[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:57 dist-test-slave-gjrz krb5kdc[760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:57 dist-test-slave-gjrz krb5kdc[760](info): set up 2 sockets
Aug 06 22:39:57 dist-test-slave-gjrz krb5kdc[760](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:39:58.430047 747 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:39:58 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:39:58.541364 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40823
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:40823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:58.624683 779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:58.624874 779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:58.624897 779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:58.625980 779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:58.626015 779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:58.626027 779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:58.626039 779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:58.626048 779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:58.627599 779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:40823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:39:58.627920 779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:58.628127 779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:58.633996 791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.634878 794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.635176 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:58.635115 779 server_base.cc:1034] running on GCE node
I20260806 22:39:58.635463 779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:58.635880 779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:58.637022 779 hybrid_clock.cc:648] HybridClock initialized: now 1786055998636981 us; error 59 us; skew 500 ppm
Aug 06 22:39:58 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:58.639777 779 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:39:58.640354 779 webserver.cc:460] Webserver started at http://127.0.186.254:39899/ using document root <none> and password file <none>
I20260806 22:39:58.640594 779 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:58.640641 779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:58.640779 779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:58.642053 779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "3694b157e21e406bbb3cb46e00ca4cb2"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "c2436fd596f0e8447699ed7e5e0b5433"
server_key_iv: "5316beb97fa0f1385cc9a49a1d7f4c53"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.642388 779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "3694b157e21e406bbb3cb46e00ca4cb2"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "c2436fd596f0e8447699ed7e5e0b5433"
server_key_iv: "5316beb97fa0f1385cc9a49a1d7f4c53"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.644172 779 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:39:58.645121 803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.645354 779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:58.645423 779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "3694b157e21e406bbb3cb46e00ca4cb2"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "c2436fd596f0e8447699ed7e5e0b5433"
server_key_iv: "5316beb97fa0f1385cc9a49a1d7f4c53"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.645489 779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:58.678818 779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:58.680837 779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:58.681030 779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:58.685873 779 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:40823
I20260806 22:39:58.686029 885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:40823 every 8 connection(s)
I20260806 22:39:58.686697 779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:58.688256 887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:58.691535 887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: Bootstrap starting.
I20260806 22:39:58.692591 887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:58.692946 887 log.cc:824] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:58.693787 887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: No bootstrap required, opened a new log
I20260806 22:39:58.695677 887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } }
I20260806 22:39:58.695855 887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:58.695967 887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3694b157e21e406bbb3cb46e00ca4cb2, State: Initialized, Role: FOLLOWER
I20260806 22:39:58.696210 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 779
I20260806 22:39:58.696146 887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } }
I20260806 22:39:58.696298 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:39:58.696287 887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:58.696333 887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:58.696388 887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:58.696904 887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } }
I20260806 22:39:58.696930 747 external_mini_cluster.cc:1428] Setting key e86945ffbcdac26e5cb3c75474217e19
I20260806 22:39:58.697106 887 leader_election.cc:302] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3694b157e21e406bbb3cb46e00ca4cb2; no voters:
I20260806 22:39:58.697317 887 leader_election.cc:288] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:58.697350 891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:58.697501 891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [term 1 LEADER]: Becoming Leader. State: Replica: 3694b157e21e406bbb3cb46e00ca4cb2, State: Running, Role: LEADER
I20260806 22:39:58.697702 891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } }
I20260806 22:39:58.697962 887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:58.698439 896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } } }
I20260806 22:39:58.698529 896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.698617 895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3694b157e21e406bbb3cb46e00ca4cb2. Latest consensus state: current_term: 1 leader_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3694b157e21e406bbb3cb46e00ca4cb2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40823 } } }
I20260806 22:39:58.698798 895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2 [sys.catalog]: This master's current role is: LEADER
W20260806 22:39:58.700767 918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:39:58.700821 918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:39:58.700964 907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:58.701292 907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:58.703437 907 catalog_manager.cc:1347] Generated new cluster ID: 80eaaf6ab0f94af8b0de62509604d38c
I20260806 22:39:58.703490 907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:39:58 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:39:58.715687 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.698241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.0.186.254:40823)
0806 22:39:58.699452 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.699461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.699500 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:39:58.701277 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.701285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.701402 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.701550 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.703884 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.705241 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.706730 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.707324 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.707408 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.711707 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.711712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.711712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.711723 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.713576 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.714098 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.714100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.714101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.714217 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.714522 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.714523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.714523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.714791 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.714930 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.715216 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.715423 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":92,"threads_started":1}
I20260806 22:39:58.730736 907 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:58.731338 907 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:58.739229 907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3694b157e21e406bbb3cb46e00ca4cb2: Generated new TSK 0
I20260806 22:39:58.739427 907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:39:58.770646 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:58.853554 926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:58.853762 926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:58.853793 926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:58.854924 926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:58.854964 926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:58.855000 926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:39:58.856477 926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:39:58.856783 926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:58.857007 926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:58.863226 937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.863315 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.863284 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:58.863476 926 server_base.cc:1034] running on GCE node
I20260806 22:39:58.863797 926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:58.864076 926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:58.865238 926 hybrid_clock.cc:648] HybridClock initialized: now 1786055998865222 us; error 31 us; skew 500 ppm
Aug 06 22:39:58 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:58.868081 926 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:39:58.868600 926 webserver.cc:460] Webserver started at http://127.0.186.193:39323/ using document root <none> and password file <none>
I20260806 22:39:58.868873 926 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:58.868952 926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:58.869095 926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:58.870424 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "85ff32516fde4e88a90c561b3abe97b3"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "e3fa7f07284b52761c4eadeab36037b2"
server_key_iv: "49c8960d32b9dc574d678d3ca3dd44ef"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.870797 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "85ff32516fde4e88a90c561b3abe97b3"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "e3fa7f07284b52761c4eadeab36037b2"
server_key_iv: "49c8960d32b9dc574d678d3ca3dd44ef"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.872344 926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:39:58.873204 946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.873380 926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:39:58.873453 926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "85ff32516fde4e88a90c561b3abe97b3"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-gjrz"
server_key: "e3fa7f07284b52761c4eadeab36037b2"
server_key_iv: "49c8960d32b9dc574d678d3ca3dd44ef"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.873529 926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:58.900770 926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:58.902580 926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:58.902709 926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:58.902998 926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:58.903401 926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:58.903491 926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.903518 926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:58.903541 926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.909536 926 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:41519
I20260806 22:39:58.909583 1116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:41519 every 8 connection(s)
I20260806 22:39:58.909880 926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:39:58 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:39:58.916674 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 926
I20260806 22:39:58.916750 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:58.916886 747 external_mini_cluster.cc:1428] Setting key c9d0552d0261785c366487c0994a1d98
I20260806 22:39:58.918912 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.910645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:50863 (local address 127.0.186.254:40823)
0806 22:39:58.910706 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.910708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.910911 (+ 203us) server_negotiation.cc:407] Connection header received
0806 22:39:58.911418 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.911419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.911427 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.911477 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.912576 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.913094 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.914070 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.914263 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.914309 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.916330 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.916332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.916332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.916339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.917638 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.918008 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.918010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.918010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.918087 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.918309 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.918310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.918310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.918423 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.918478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.918736 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.918825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:39:58.919070 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.910611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40823 (local address 127.0.186.193:50863)
0806 22:39:58.910781 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.910824 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.911331 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.911541 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.911545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.912286 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.912482 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.912489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.913179 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.913179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.913974 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.913976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.914324 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.914324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.914425 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.914834 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.914842 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.916227 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.917717 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.917725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.917727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.917938 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.918146 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.918146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.918147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.918251 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.918477 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.918484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.918619 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.918706 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.918858 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260806 22:39:58.919957 1117 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40823
I20260806 22:39:58.920064 1117 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:58.920264 1117 heartbeater.cc:511] Master 127.0.186.254:40823 requested a full tablet report, sending...
I20260806 22:39:58.921113 826 ts_manager.cc:194] Registered new tserver with Master: 85ff32516fde4e88a90c561b3abe97b3 (127.0.186.193:41519)
I20260806 22:39:58.921855 826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:50863
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:39:58.950258 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:59.067466 1124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:59.067667 1124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:59.067700 1124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:59.068945 1124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:59.068988 1124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:59.069024 1124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:39:59.070705 1124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:39:59.070958 1124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:59.071169 1124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:59.077883 1136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:59.077924 1132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:59.077993 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:59.078315 1124 server_base.cc:1034] running on GCE node
I20260806 22:39:59.078599 1124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:59.078842 1124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:59.080022 1124 hybrid_clock.cc:648] HybridClock initialized: now 1786055999080005 us; error 27 us; skew 500 ppm
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:59.083474 1124 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:39:59.083868 1124 webserver.cc:460] Webserver started at http://127.0.186.194:32947/ using document root <none> and password file <none>
I20260806 22:39:59.084088 1124 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:59.084137 1124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:59.084256 1124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:59.085283 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "9a2cee77a69a4055be8431b675bbd223"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "e51a1c103bf205d35a6f069907d1b8c0"
server_key_iv: "48966eb1530c908de3d750d9b1062bbe"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.085660 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "9a2cee77a69a4055be8431b675bbd223"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "e51a1c103bf205d35a6f069907d1b8c0"
server_key_iv: "48966eb1530c908de3d750d9b1062bbe"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.087235 1124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:39:59.087997 1147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.088140 1124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:39:59.088217 1124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "9a2cee77a69a4055be8431b675bbd223"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "e51a1c103bf205d35a6f069907d1b8c0"
server_key_iv: "48966eb1530c908de3d750d9b1062bbe"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.088282 1124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:59.104182 1124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:59.106601 1124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:59.106808 1124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:59.107213 1124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:59.107630 1124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:59.107667 1124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.107697 1124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:59.107712 1124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.120283 1124 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34351
I20260806 22:39:59.120379 1331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34351 every 8 connection(s)
I20260806 22:39:59.120673 1124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:59.126108 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1124
I20260806 22:39:59.126211 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:59.126371 747 external_mini_cluster.cc:1428] Setting key cf30363a11d82ff970452cb32dfb92ea
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:39:59.129909 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.121411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:42875 (local address 127.0.186.254:40823)
0806 22:39:59.121473 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:59.121475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:59.121730 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:39:59.122220 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:59.122221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:59.122232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:59.122282 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:59.123174 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.123716 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.124795 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.125020 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.125085 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.127345 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:59.127347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:59.127347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:59.127354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:59.128650 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.129028 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.129030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.129030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.129088 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.129307 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.129307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.129308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.129422 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:59.129478 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:59.129730 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:39:59.129821 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:39:59.130129 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.121375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40823 (local address 127.0.186.194:42875)
0806 22:39:59.121577 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:59.121627 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:59.122137 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:59.122335 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:59.122338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:59.122908 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:59.123106 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:59.123111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.123818 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.123819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:59.124649 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:59.124652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.125114 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.125116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:59.125230 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.125692 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:59.125700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:59.127244 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:59.128725 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:59.128733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:59.128735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:59.128960 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:59.129141 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:59.129142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:59.129143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:59.129252 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:59.129476 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:59.129483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:59.129617 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 22:39:59.129703 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:39:59.129866 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":136,"threads_started":1}
I20260806 22:39:59.130620 1332 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40823
I20260806 22:39:59.130704 1332 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:59.130892 1332 heartbeater.cc:511] Master 127.0.186.254:40823 requested a full tablet report, sending...
I20260806 22:39:59.131283 826 ts_manager.cc:194] Registered new tserver with Master: 9a2cee77a69a4055be8431b675bbd223 (127.0.186.194:34351)
I20260806 22:39:59.131764 826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:42875
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:39:59.153360 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:59.264087 1337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:59.264283 1337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:59.264307 1337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:59.265501 1337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:59.265542 1337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:59.265568 1337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:39:59.267199 1337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:39:59.267452 1337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:59.267661 1337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:59.273217 1347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:59.273255 1350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:59.273308 1348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:59.273679 1337 server_base.cc:1034] running on GCE node
I20260806 22:39:59.273893 1337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:59.274112 1337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:59.275250 1337 hybrid_clock.cc:648] HybridClock initialized: now 1786055999275234 us; error 27 us; skew 500 ppm
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:59.277724 1337 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:39:59.278129 1337 webserver.cc:460] Webserver started at http://127.0.186.195:39261/ using document root <none> and password file <none>
I20260806 22:39:59.278344 1337 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:59.278391 1337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:59.278524 1337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:59.279578 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "59aae0a732cb48cb9b685bd1d1049147"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "3d2deed66be89a610e36b9af80b53b46"
server_key_iv: "f24c60ddd2430abb07320485a22ec8cc"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.279906 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "59aae0a732cb48cb9b685bd1d1049147"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "3d2deed66be89a610e36b9af80b53b46"
server_key_iv: "f24c60ddd2430abb07320485a22ec8cc"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.281203 1337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:39:59.281891 1358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.282037 1337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.282119 1337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "59aae0a732cb48cb9b685bd1d1049147"
format_stamp: "Formatted at 2026-08-06 22:39:59 on dist-test-slave-gjrz"
server_key: "3d2deed66be89a610e36b9af80b53b46"
server_key_iv: "f24c60ddd2430abb07320485a22ec8cc"
server_key_version: "encryptionkey@0"
I20260806 22:39:59.282176 1337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:59.292598 1337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:59.294194 1337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:59.294322 1337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:59.294525 1337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:59.294800 1337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:59.294832 1337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.294858 1337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:59.294883 1337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:59.309401 1337 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:38729
I20260806 22:39:59.309532 1548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:38729 every 8 connection(s)
I20260806 22:39:59.309775 1337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:39:59.318818 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1337
I20260806 22:39:59.318898 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:59.319067 747 external_mini_cluster.cc:1428] Setting key 1707c4fc41c2b04b241c9385aa9f116c
I20260806 22:39:59.319490 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.310532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:60495 (local address 127.0.186.254:40823)
0806 22:39:59.310600 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:59.310602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:59.310953 (+ 351us) server_negotiation.cc:407] Connection header received
0806 22:39:59.311773 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:59.311774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:59.311784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:59.311833 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:59.312910 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.313403 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.314549 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.314741 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.314791 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.317053 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:59.317054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:59.317054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:59.317061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:59.318275 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.318611 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.318612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.318612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.318659 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.318844 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.318845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.318845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.318979 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:59.319045 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:59.319324 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:39:59.319406 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:39:59.319743 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.310478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40823 (local address 127.0.186.195:60495)
0806 22:39:59.310782 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:59.310832 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:59.311684 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:59.311889 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:59.311892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:59.312517 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:59.312820 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:59.312826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.313503 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.313504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:59.314467 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:59.314470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.314801 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.314802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:59.314915 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.315468 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:59.315477 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:59.316968 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:59.318340 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:59.318345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:59.318346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:59.318556 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:59.318709 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:59.318709 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:59.318709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:59.318794 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:59.319039 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:59.319044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:59.319176 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:39:59.319258 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:39:59.319448 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":163,"threads_started":1}
I20260806 22:39:59.320191 1549 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40823
I20260806 22:39:59.320271 1549 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:59.320432 1549 heartbeater.cc:511] Master 127.0.186.254:40823 requested a full tablet report, sending...
I20260806 22:39:59.320768 826 ts_manager.cc:194] Registered new tserver with Master: 59aae0a732cb48cb9b685bd1d1049147 (127.0.186.195:38729)
I20260806 22:39:59.321144 826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:60495
I20260806 22:39:59.331167 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:39:59.356292 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.348862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.0.186.193:41519)
0806 22:39:59.349087 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:59.349092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:59.349102 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:39:59.349130 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:59.349133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:59.349149 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:59.349225 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:59.349859 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.350554 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.351336 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.351637 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.351686 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.353883 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:59.353890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:59.353890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:59.353896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:59.355227 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.355646 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.355648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.355648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.355716 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.355843 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.355844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.355844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.355996 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:59.356104 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:59.356143 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:39:59.356203 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":149,"threads_started":1}
W20260806 22:39:59.356739 993 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59838
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:39:59.380051 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:59.373445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.0.186.193:41519)
0806 22:39:59.373520 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:59.373522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:59.373561 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:39:59.373626 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:59.373627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:59.373640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:59.373687 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:59.374247 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.374791 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.375691 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:59.375959 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:59.376019 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:59.378048 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:59.378050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:59.378050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:59.378059 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:59.379063 (+ 1004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.379376 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.379378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.379378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.379428 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:59.379594 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:59.379595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:59.379595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:59.379685 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:59.379852 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:59.379888 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:39:59.379964 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:39:59.380493 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 926
I20260806 22:39:59.393682 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1124
I20260806 22:39:59.402009 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1337
I20260806 22:39:59.410413 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 779
2026-08-06T22:39:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2178 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[1570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[1570](info): set up 2 sockets
Aug 06 22:39:59 dist-test-slave-gjrz krb5kdc[1570](info): commencing operation
krb5kdc: starting...
W20260806 22:40:00.547784 747 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:00 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:00.661697 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:43219
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:00.740971 1584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:00.741160 1584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:00.741181 1584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:00.742291 1584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:00.742331 1584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:00.742344 1584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:00.742355 1584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:00.742376 1584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:00.743834 1584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:43219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:43219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:00.744045 1584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:00.744252 1584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:00.749001 1598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.749089 1584 server_base.cc:1034] running on GCE node
W20260806 22:40:00.749011 1597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.749013 1603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.749465 1584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:00.749763 1584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:00.750905 1584 hybrid_clock.cc:648] HybridClock initialized: now 1786056000750893 us; error 28 us; skew 500 ppm
Aug 06 22:40:00 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:00.753280 1584 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:00.753696 1584 webserver.cc:460] Webserver started at http://127.0.186.254:36343/ using document root <none> and password file <none>
I20260806 22:40:00.753885 1584 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:00.753930 1584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:00.754050 1584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:00.755065 1584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "135b4b82cfd64f129aa2b3aed0f5a32d"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "535b02c0714804f354054562410cd0a7"
server_key_iv: "59229eca302fe2b2d8ad4cd5ebcdaf8a"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.755434 1584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "135b4b82cfd64f129aa2b3aed0f5a32d"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "535b02c0714804f354054562410cd0a7"
server_key_iv: "59229eca302fe2b2d8ad4cd5ebcdaf8a"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.756614 1584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:00.757251 1618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.757395 1584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.757483 1584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "135b4b82cfd64f129aa2b3aed0f5a32d"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "535b02c0714804f354054562410cd0a7"
server_key_iv: "59229eca302fe2b2d8ad4cd5ebcdaf8a"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.757546 1584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:00.776768 1584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:00.778496 1584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:00.778635 1584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:00.782498 1584 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:43219
I20260806 22:40:00.782545 1698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:43219 every 8 connection(s)
I20260806 22:40:00.782882 1584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:00.783746 1699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:00.785876 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1584
I20260806 22:40:00.785953 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:00.786010 1699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d: Bootstrap starting.
I20260806 22:40:00.786103 747 external_mini_cluster.cc:1428] Setting key 797128ea5b622ed97e2f6f486b26fa8d
I20260806 22:40:00.786718 1699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:00.787036 1699 log.cc:824] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:00.787812 1699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d: No bootstrap required, opened a new log
I20260806 22:40:00.789599 1699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } }
I20260806 22:40:00.789789 1699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:00.789827 1699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135b4b82cfd64f129aa2b3aed0f5a32d, State: Initialized, Role: FOLLOWER
I20260806 22:40:00.789963 1699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } }
I20260806 22:40:00.790047 1699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:00.790081 1699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:00.790138 1699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:00.790758 1699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } }
I20260806 22:40:00.790851 1699 leader_election.cc:302] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135b4b82cfd64f129aa2b3aed0f5a32d; no voters:
I20260806 22:40:00.791038 1699 leader_election.cc:288] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:00.791085 1706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:00.791249 1706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [term 1 LEADER]: Becoming Leader. State: Replica: 135b4b82cfd64f129aa2b3aed0f5a32d, State: Running, Role: LEADER
I20260806 22:40:00.791349 1699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:00.791412 1706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } }
I20260806 22:40:00.791754 1706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 135b4b82cfd64f129aa2b3aed0f5a32d. Latest consensus state: current_term: 1 leader_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } } }
I20260806 22:40:00.791837 1706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:00.791985 1709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135b4b82cfd64f129aa2b3aed0f5a32d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 43219 } } }
I20260806 22:40:00.792045 1709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:00.792177 1720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:00.792979 1720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:40:00 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:00.796059 1720 catalog_manager.cc:1347] Generated new cluster ID: d669a2640c7d414aab26cabdc1b5cebb
I20260806 22:40:00.796136 1720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:00.797950 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:00.786604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.0.186.254:43219)
0806 22:40:00.786795 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:00.786805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:00.786842 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:40:00.787427 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:00.787435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:00.787489 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:00.787639 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:00.788265 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.790453 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.791384 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.791994 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.792100 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:00.794765 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:00.794769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:00.794769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:00.794779 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:00.796548 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.796878 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.796881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.796881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.796989 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.797174 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.797175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.797175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.797397 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:00.797540 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:00.797647 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:40:00.797779 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260806 22:40:00.808130 1720 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:00.808842 1720 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:00.814364 1720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 135b4b82cfd64f129aa2b3aed0f5a32d: Generated new TSK 0
I20260806 22:40:00.814527 1720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:00.847398 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:00.941967 1742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:00.942173 1742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:00.942199 1742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:00.943378 1742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:00.943418 1742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:00.943444 1742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:00.944852 1742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:00.945068 1742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:00.945276 1742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:00.949916 1748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.949950 1751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.950006 1755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.950122 1742 server_base.cc:1034] running on GCE node
I20260806 22:40:00.950408 1742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:00.950629 1742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:00.951761 1742 hybrid_clock.cc:648] HybridClock initialized: now 1786056000951746 us; error 28 us; skew 500 ppm
Aug 06 22:40:00 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:00.954021 1742 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:00.954416 1742 webserver.cc:460] Webserver started at http://127.0.186.193:35661/ using document root <none> and password file <none>
I20260806 22:40:00.954602 1742 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:00.954649 1742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:00.954773 1742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:00.955772 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "6f974c5beedc4092b561dc67a3558f63"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "444eaa7522295884a8c83791cb36a328"
server_key_iv: "eb67dc006996e03f551d3d64d5dc7f1b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.956095 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "6f974c5beedc4092b561dc67a3558f63"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "444eaa7522295884a8c83791cb36a328"
server_key_iv: "eb67dc006996e03f551d3d64d5dc7f1b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.957253 1742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:00.957876 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.958015 1742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:00.958096 1742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "6f974c5beedc4092b561dc67a3558f63"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-gjrz"
server_key: "444eaa7522295884a8c83791cb36a328"
server_key_iv: "eb67dc006996e03f551d3d64d5dc7f1b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.958153 1742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:00.982776 1742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:00.984541 1742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:00.984673 1742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:00.984877 1742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:00.985186 1742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:00.985219 1742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.985248 1742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:00.985265 1742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.990556 1742 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:45559
I20260806 22:40:00.990618 1946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:45559 every 8 connection(s)
I20260806 22:40:00.990911 1742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:00.992965 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1742
I20260806 22:40:00.993052 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:00.993220 747 external_mini_cluster.cc:1428] Setting key 6e64805f080372ae82e21dbbe11c8902
Aug 06 22:40:00 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:01.000620 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:00.991619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:33461 (local address 127.0.186.254:43219)
0806 22:40:00.991679 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:00.991681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:00.991924 (+ 243us) server_negotiation.cc:407] Connection header received
0806 22:40:00.992390 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:00.992391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:00.992401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:00.992448 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:00.993575 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.994168 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.995472 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.995771 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.995843 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:00.997952 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:00.997954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:00.997954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:00.997960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:00.999254 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.999713 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.999714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.999714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.999763 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.000024 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.000025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.000025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.000129 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.000177 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.000487 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.000556 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:40:01.000849 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:00.991589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:43219 (local address 127.0.186.193:33461)
0806 22:40:00.991804 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:00.991847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:00.992305 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:00.992506 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:00.992509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:00.993198 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:00.993475 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:00.993483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.994258 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.994259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:00.995388 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:00.995392 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.995848 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.995849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:00.995948 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:00.996385 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:00.996393 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:00.997866 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:00.999348 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:00.999356 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:00.999359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:00.999645 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:00.999819 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:00.999819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:00.999820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:00.999955 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:01.000185 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:01.000191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:01.000353 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.000470 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.000633 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260806 22:40:01.001338 1948 heartbeater.cc:347] Connected to a master server at 127.0.186.254:43219
I20260806 22:40:01.001434 1948 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:01.001621 1948 heartbeater.cc:511] Master 127.0.186.254:43219 requested a full tablet report, sending...
I20260806 22:40:01.002153 1646 ts_manager.cc:194] Registered new tserver with Master: 6f974c5beedc4092b561dc67a3558f63 (127.0.186.193:45559)
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
I20260806 22:40:01.002738 1646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:33461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:01.021575 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:01.105208 1954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:01.105422 1954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:01.105454 1954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:01.106622 1954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:01.106666 1954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:01.106702 1954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:01.108178 1954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:01.108428 1954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:01.108641 1954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:01.113348 1961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:01.113359 1965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:01.113356 1963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:01.113433 1954 server_base.cc:1034] running on GCE node
I20260806 22:40:01.113721 1954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:01.113960 1954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:01.115098 1954 hybrid_clock.cc:648] HybridClock initialized: now 1786056001115085 us; error 29 us; skew 500 ppm
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:01.117424 1954 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:01.117861 1954 webserver.cc:460] Webserver started at http://127.0.186.194:35491/ using document root <none> and password file <none>
I20260806 22:40:01.118062 1954 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:01.118106 1954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:01.118228 1954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:01.119238 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "488ae7240671461a88f94f878a8a0bb3"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "0a1a933fbac061439b7f119cefe09ed9"
server_key_iv: "48f9db37625bd2326b211fc8f8700d4a"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.119629 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "488ae7240671461a88f94f878a8a0bb3"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "0a1a933fbac061439b7f119cefe09ed9"
server_key_iv: "48f9db37625bd2326b211fc8f8700d4a"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.120849 1954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:01.121558 1974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.121762 1954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:01.121852 1954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "488ae7240671461a88f94f878a8a0bb3"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "0a1a933fbac061439b7f119cefe09ed9"
server_key_iv: "48f9db37625bd2326b211fc8f8700d4a"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.121945 1954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:01.151005 1954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:01.152938 1954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:01.153050 1954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:01.153282 1954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:01.153599 1954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:01.153635 1954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.153663 1954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:01.153687 1954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.165086 1954 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:46417
I20260806 22:40:01.165146 2160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:46417 every 8 connection(s)
I20260806 22:40:01.165468 1954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:01.166785 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1954
I20260806 22:40:01.166855 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:01.167013 747 external_mini_cluster.cc:1428] Setting key 2030b91590ea4b69b1553bb6c5cab4f3
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:01.176131 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.166158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50483 (local address 127.0.186.254:43219)
0806 22:40:01.166201 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.166203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.166456 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:40:01.167041 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.167042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.167056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.167117 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.168635 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.169255 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.170552 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.170832 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.170908 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.173458 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.173460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.173460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.173466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.174719 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.175172 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.175173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.175173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.175219 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.175502 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.175503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.175503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.175624 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.175677 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.175984 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.176065 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:01.176384 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.166142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:43219 (local address 127.0.186.194:50483)
0806 22:40:01.166337 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.166374 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.166941 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.167209 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.167213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.168095 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:01.168529 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.168537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.169361 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.169363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.170460 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.170464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.170919 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.170920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.171069 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.171582 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:01.171591 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:01.173343 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:01.174799 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:01.174806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:01.174808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:01.175105 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:01.175294 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:01.175294 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:01.175295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:01.175441 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:01.175687 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:01.175693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:01.175851 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.175963 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.176131 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
I20260806 22:40:01.176944 2161 heartbeater.cc:347] Connected to a master server at 127.0.186.254:43219
I20260806 22:40:01.177052 2161 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:01.177225 2161 heartbeater.cc:511] Master 127.0.186.254:43219 requested a full tablet report, sending...
I20260806 22:40:01.177588 1646 ts_manager.cc:194] Registered new tserver with Master: 488ae7240671461a88f94f878a8a0bb3 (127.0.186.194:46417)
I20260806 22:40:01.177971 1646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:50483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:01.195346 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:01.299207 2171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:01.299444 2171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:01.299479 2171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:01.300631 2171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:01.300670 2171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:01.300705 2171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:01.302175 2171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:43219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:01.302423 2171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:01.302632 2171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:01.308081 2183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:01.308164 2177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:01.308091 2179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:01.308303 2171 server_base.cc:1034] running on GCE node
I20260806 22:40:01.308507 2171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:01.308759 2171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:01.309934 2171 hybrid_clock.cc:648] HybridClock initialized: now 1786056001309914 us; error 32 us; skew 500 ppm
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:01.312978 2171 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:01.313378 2171 webserver.cc:460] Webserver started at http://127.0.186.195:37551/ using document root <none> and password file <none>
I20260806 22:40:01.313592 2171 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:01.313637 2171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:01.313761 2171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:01.314842 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "f3fe0d6ee10b4717b2d77224b7acf864"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "5b8be3485223fc582b19691458e832b9"
server_key_iv: "039493a5a0c6ffd5bba6731ec3eef04d"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.315176 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "f3fe0d6ee10b4717b2d77224b7acf864"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "5b8be3485223fc582b19691458e832b9"
server_key_iv: "039493a5a0c6ffd5bba6731ec3eef04d"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.318076 2171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:01.318790 2191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.318964 2171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:01.319044 2171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "f3fe0d6ee10b4717b2d77224b7acf864"
format_stamp: "Formatted at 2026-08-06 22:40:01 on dist-test-slave-gjrz"
server_key: "5b8be3485223fc582b19691458e832b9"
server_key_iv: "039493a5a0c6ffd5bba6731ec3eef04d"
server_key_version: "encryptionkey@0"
I20260806 22:40:01.319111 2171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:01.340425 2171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:01.342401 2171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:01.342530 2171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:01.342777 2171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:01.343091 2171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:01.343127 2171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.343148 2171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:01.343159 2171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:01.349223 2171 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:38855
I20260806 22:40:01.349301 2366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:38855 every 8 connection(s)
I20260806 22:40:01.349563 2171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:01.350601 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2171
I20260806 22:40:01.350684 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:01.350833 747 external_mini_cluster.cc:1428] Setting key 71a1c9627809d6720133433e72c21893
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:01.358529 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.350282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:34427 (local address 127.0.186.254:43219)
0806 22:40:01.350329 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.350332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.350545 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:40:01.351180 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.351181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.351192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.351247 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.352339 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.352834 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.353800 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.354003 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.354058 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.356084 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.356086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.356086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.356094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.357294 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.357663 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.357664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.357665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.357720 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.357933 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.357934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.357934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.358050 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.358108 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.358361 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.358438 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:01.358705 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.350264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:43219 (local address 127.0.186.195:34427)
0806 22:40:01.350415 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.350460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.351032 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.351371 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.351375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.352057 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:01.352251 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.352256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.352913 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.352914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.353721 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.353724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.354071 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.354072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.354203 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.354644 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:01.354653 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:01.355996 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:01.357369 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:01.357376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:01.357379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:01.357597 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:01.357774 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:01.357775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:01.357775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:01.357878 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:01.358106 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:01.358112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:01.358244 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.358333 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.358497 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260806 22:40:01.359153 2369 heartbeater.cc:347] Connected to a master server at 127.0.186.254:43219
I20260806 22:40:01.359241 2369 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:01.359439 2369 heartbeater.cc:511] Master 127.0.186.254:43219 requested a full tablet report, sending...
I20260806 22:40:01.359809 1646 ts_manager.cc:194] Registered new tserver with Master: f3fe0d6ee10b4717b2d77224b7acf864 (127.0.186.195:38855)
I20260806 22:40:01.360174 1646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:34427
I20260806 22:40:01.362417 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:01.369560 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.364256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.0.186.254:43219)
0806 22:40:01.364313 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.364315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.364392 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:40:01.364448 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.364449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.364458 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.364504 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.365055 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.365506 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.366195 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.366381 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.366434 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.367533 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.367534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.367534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.367541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.368556 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.368855 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.368856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.368856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.368907 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.369063 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.369064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.369064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.369198 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.369291 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.369398 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.369469 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:01.372017 1646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:01.373867 1646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:01.380623 2397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.377123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.0.186.195:38855)
0806 22:40:01.377286 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.377290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.377299 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:01.377331 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.377333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.377347 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.377416 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:01.377621 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378662 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.379653 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380250 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380313 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380369 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.380512 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.380552 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260806 22:40:01.380614 2393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.376890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:38855 (local address 127.0.0.1:41008)
0806 22:40:01.377150 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.377157 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.377196 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.377490 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.377491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.377493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:01.377538 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.377540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378750 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.378751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.379565 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.379567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380330 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.380411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380426 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.380487 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.380541 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":72,"threads_started":1}
I20260806 22:40:01.381003 2391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.376681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:46417 (local address 127.0.0.1:50434)
0806 22:40:01.376898 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.376982 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.377053 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.377381 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.377382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.377384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:01.377433 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.377435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378439 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.378440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.379880 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.379882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380730 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.380796 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380800 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.380858 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.380896 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":34,"thread_start_us":51,"threads_started":1}
I20260806 22:40:01.381006 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.376807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.0.186.194:46417)
0806 22:40:01.377183 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.377186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.377195 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:01.377241 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.377243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.377259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.377389 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:01.377490 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378342 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.379961 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380626 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380719 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380768 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.380887 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.380929 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":98,"threads_started":1}
I20260806 22:40:01.381016 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.376688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:45559 (local address 127.0.0.1:39776)
0806 22:40:01.377009 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.377017 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.377055 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.377277 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.377280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.377281 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:01.377344 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.377345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378622 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.378623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.379756 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.379759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380761 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.380826 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380830 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.380877 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.380921 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":33,"thread_start_us":82,"threads_started":1}
I20260806 22:40:01.381176 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.376903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.0.186.193:45559)
0806 22:40:01.377050 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.377055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.377067 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:01.377185 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.377189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.377207 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.377277 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:01.377423 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.378510 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.379852 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.380593 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.380664 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.380726 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.381057 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.381107 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260806 22:40:01.381690 2044 tablet_service.cc:1467] Processing CreateTablet for tablet a0524b67ae74443cbefbdffa1d4799e7 (DEFAULT_TABLE table=test-table [id=b5025ef066864478b85104ea85518db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:01.381704 2258 tablet_service.cc:1467] Processing CreateTablet for tablet a0524b67ae74443cbefbdffa1d4799e7 (DEFAULT_TABLE table=test-table [id=b5025ef066864478b85104ea85518db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:01.382047 2044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0524b67ae74443cbefbdffa1d4799e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:01.382071 2258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0524b67ae74443cbefbdffa1d4799e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:01.384684 2399 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Bootstrap starting.
I20260806 22:40:01.384927 2398 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: Bootstrap starting.
I20260806 22:40:01.385182 1836 tablet_service.cc:1467] Processing CreateTablet for tablet a0524b67ae74443cbefbdffa1d4799e7 (DEFAULT_TABLE table=test-table [id=b5025ef066864478b85104ea85518db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:01.385387 2399 tablet_bootstrap.cc:654] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:01.385505 2398 tablet_bootstrap.cc:654] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:01.385526 1836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0524b67ae74443cbefbdffa1d4799e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:01.386214 2399 log.cc:824] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:01.386994 2399 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: No bootstrap required, opened a new log
I20260806 22:40:01.387072 2399 ts_tablet_manager.cc:1397] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:01.387674 2398 log.cc:824] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:01.388306 2398 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: No bootstrap required, opened a new log
I20260806 22:40:01.388375 2398 ts_tablet_manager.cc:1397] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:40:01.389137 2399 raft_consensus.cc:348] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.389338 2399 raft_consensus.cc:374] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:01.389395 2399 raft_consensus.cc:729] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488ae7240671461a88f94f878a8a0bb3, State: Initialized, Role: FOLLOWER
I20260806 22:40:01.389529 2399 consensus_queue.cc:260] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.389755 2404 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: Bootstrap starting.
I20260806 22:40:01.389808 2399 ts_tablet_manager.cc:1428] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:01.389978 2161 heartbeater.cc:503] Master 127.0.186.254:43219 was elected leader, sending a full tablet report...
I20260806 22:40:01.390501 2404 tablet_bootstrap.cc:654] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:01.389883 2398 raft_consensus.cc:348] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.390551 2398 raft_consensus.cc:374] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:01.390601 2398 raft_consensus.cc:729] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3fe0d6ee10b4717b2d77224b7acf864, State: Initialized, Role: FOLLOWER
I20260806 22:40:01.390774 2398 consensus_queue.cc:260] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.391065 2398 ts_tablet_manager.cc:1428] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:01.391052 2404 log.cc:824] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:01.391176 2369 heartbeater.cc:503] Master 127.0.186.254:43219 was elected leader, sending a full tablet report...
I20260806 22:40:01.391903 2404 tablet_bootstrap.cc:492] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: No bootstrap required, opened a new log
I20260806 22:40:01.391964 2404 ts_tablet_manager.cc:1397] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:01.393337 2404 raft_consensus.cc:348] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.393468 2404 raft_consensus.cc:374] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:01.393489 2404 raft_consensus.cc:729] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f974c5beedc4092b561dc67a3558f63, State: Initialized, Role: FOLLOWER
I20260806 22:40:01.393589 2404 consensus_queue.cc:260] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.393867 2404 ts_tablet_manager.cc:1428] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:01.393888 1948 heartbeater.cc:503] Master 127.0.186.254:43219 was elected leader, sending a full tablet report...
W20260806 22:40:01.416077 2162 tablet.cc:2369] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:01.417999 2406 raft_consensus.cc:482] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:01.418118 2406 raft_consensus.cc:504] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.418747 2406 leader_election.cc:288] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f974c5beedc4092b561dc67a3558f63 (127.0.186.193:45559), 488ae7240671461a88f94f878a8a0bb3 (127.0.186.194:46417)
I20260806 22:40:01.421675 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.418914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:45559 (local address 127.0.186.195:34547)
0806 22:40:01.418973 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.418987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.419033 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.419218 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.419219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.419221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:01.419313 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.419315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.419993 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.419994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.420792 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.420795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.421398 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.421400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.421516 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.421520 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.421572 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.421609 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260806 22:40:01.421690 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.419031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:34547 (local address 127.0.186.193:45559)
0806 22:40:01.419080 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.419082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.419090 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:01.419117 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.419118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.419128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.419200 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:01.419399 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.419910 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.420889 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.421322 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.421379 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.421387 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.421588 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.421627 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:01.421761 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.418958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:46417 (local address 127.0.186.195:43525)
0806 22:40:01.419146 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:01.419156 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:01.419211 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:01.419378 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:01.419379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:01.419380 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:01.419450 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.419452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.420136 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.420137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.420893 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:01.420895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.421465 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.421466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:01.421523 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.421526 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:01.421576 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:40:01.421617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":117,"threads_started":1}
I20260806 22:40:01.421823 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.419071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:43525 (local address 127.0.186.194:46417)
0806 22:40:01.419124 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.419126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.419198 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:40:01.419299 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.419300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.419311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.419368 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:01.419552 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.420041 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.420982 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.421391 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.421459 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.421468 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.421721 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.421764 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:01.422187 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0524b67ae74443cbefbdffa1d4799e7" candidate_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "488ae7240671461a88f94f878a8a0bb3" is_pre_election: true
I20260806 22:40:01.422379 2077 raft_consensus.cc:2454] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3fe0d6ee10b4717b2d77224b7acf864 in term 0.
I20260806 22:40:01.422389 1867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0524b67ae74443cbefbdffa1d4799e7" candidate_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f974c5beedc4092b561dc67a3558f63" is_pre_election: true
I20260806 22:40:01.422524 1867 raft_consensus.cc:2454] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3fe0d6ee10b4717b2d77224b7acf864 in term 0.
I20260806 22:40:01.422696 2192 leader_election.cc:302] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 488ae7240671461a88f94f878a8a0bb3, f3fe0d6ee10b4717b2d77224b7acf864; no voters:
I20260806 22:40:01.422832 2406 raft_consensus.cc:2781] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:01.422888 2406 raft_consensus.cc:482] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:01.422914 2406 raft_consensus.cc:3037] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:01.423689 2406 raft_consensus.cc:504] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.423804 2406 leader_election.cc:288] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [CANDIDATE]: Term 1 election: Requested vote from peers 6f974c5beedc4092b561dc67a3558f63 (127.0.186.193:45559), 488ae7240671461a88f94f878a8a0bb3 (127.0.186.194:46417)
I20260806 22:40:01.423985 1867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0524b67ae74443cbefbdffa1d4799e7" candidate_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f974c5beedc4092b561dc67a3558f63"
I20260806 22:40:01.423988 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0524b67ae74443cbefbdffa1d4799e7" candidate_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "488ae7240671461a88f94f878a8a0bb3"
I20260806 22:40:01.424059 2077 raft_consensus.cc:3037] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:01.424062 1867 raft_consensus.cc:3037] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:01.424785 2077 raft_consensus.cc:2454] T a0524b67ae74443cbefbdffa1d4799e7 P 488ae7240671461a88f94f878a8a0bb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3fe0d6ee10b4717b2d77224b7acf864 in term 1.
I20260806 22:40:01.424899 1867 raft_consensus.cc:2454] T a0524b67ae74443cbefbdffa1d4799e7 P 6f974c5beedc4092b561dc67a3558f63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3fe0d6ee10b4717b2d77224b7acf864 in term 1.
I20260806 22:40:01.424958 2192 leader_election.cc:302] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 488ae7240671461a88f94f878a8a0bb3, f3fe0d6ee10b4717b2d77224b7acf864; no voters:
I20260806 22:40:01.425047 2406 raft_consensus.cc:2781] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:01.425199 2406 raft_consensus.cc:686] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [term 1 LEADER]: Becoming Leader. State: Replica: f3fe0d6ee10b4717b2d77224b7acf864, State: Running, Role: LEADER
I20260806 22:40:01.425315 2406 consensus_queue.cc:237] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } }
I20260806 22:40:01.426195 1646 catalog_manager.cc:5495] T a0524b67ae74443cbefbdffa1d4799e7 P f3fe0d6ee10b4717b2d77224b7acf864 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3fe0d6ee10b4717b2d77224b7acf864 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f974c5beedc4092b561dc67a3558f63" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3fe0d6ee10b4717b2d77224b7acf864" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "488ae7240671461a88f94f878a8a0bb3" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46417 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:01.434532 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.427849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.0.186.193:45559)
0806 22:40:01.427920 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.427922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.427986 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:40:01.428037 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.428037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.428045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.428087 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.428623 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.429075 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.429756 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.429940 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.429991 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.432242 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.432250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.432250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.432257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.433570 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.433881 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.433884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.433885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.433951 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.434090 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.434090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.434091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.434213 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.434312 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.434401 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.434471 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:01.458629 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.452059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.0.186.193:45559)
0806 22:40:01.452143 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.452145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.452192 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:40:01.452256 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.452257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.452267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.452310 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.452847 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.453419 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.454132 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.454375 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.454434 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.456558 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.456560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.456560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.456567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.457596 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.457919 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.457920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.457920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.457972 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.458135 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.458136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.458136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.458226 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.458314 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.458473 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.458542 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[1570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:01.481061 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.474625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39820 (local address 127.0.186.193:45559)
0806 22:40:01.474685 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.474687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.474744 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:40:01.474811 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.474812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.474823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.474868 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.475412 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.475870 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.476609 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.476793 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.476855 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.479034 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.479035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.479036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.479042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.480157 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.480501 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.480503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.480503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.480559 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.480699 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.480699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.480700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.480785 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.480851 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.480926 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.480990 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 22:40:01.481822 1805 tablet_service.cc:2908] Rejecting scan request for tablet a0524b67ae74443cbefbdffa1d4799e7: Uninitialized: clean time has not yet been initialized
I20260806 22:40:01.483084 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1742
I20260806 22:40:01.493201 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1954
I20260806 22:40:01.502260 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2171
I20260806 22:40:01.507151 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1584
2026-08-06T22:40:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2099 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[2438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[2438](info): set up 2 sockets
Aug 06 22:40:01 dist-test-slave-gjrz krb5kdc[2438](info): commencing operation
krb5kdc: starting...
W20260806 22:40:02.658885 747 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:02 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:02.777978 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45517
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:45517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:02.859320 2458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:02.859519 2458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:02.859540 2458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:02.860713 2458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:02.860757 2458 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:40:02.860769 2458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:02.860782 2458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:02.860805 2458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:02.860828 2458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:02.862349 2458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:45517
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:02.862572 2458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:02.862790 2458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:02.867429 2464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:02.867537 2471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.867698 2458 server_base.cc:1034] running on GCE node
W20260806 22:40:02.867429 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.867976 2458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:02.868256 2458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:02.869412 2458 hybrid_clock.cc:648] HybridClock initialized: now 1786056002869399 us; error 30 us; skew 500 ppm
Aug 06 22:40:02 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:02.875396 2458 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:02.875804 2458 webserver.cc:460] Webserver started at http://127.0.186.254:39713/ using document root <none> and password file <none>
I20260806 22:40:02.875995 2458 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:02.876041 2458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:02.876163 2458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:02.877152 2458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "118a031da8824527a15e8747b74df7a4"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-gjrz"
server_key: "63576fe84517c792262470c99fcff4e0"
server_key_iv: "200785ab36ca9d63a2b49ed958834ecf"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.877478 2458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "118a031da8824527a15e8747b74df7a4"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-gjrz"
server_key: "63576fe84517c792262470c99fcff4e0"
server_key_iv: "200785ab36ca9d63a2b49ed958834ecf"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.878700 2458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:02.879381 2485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:02.879521 2458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:02.879611 2458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "118a031da8824527a15e8747b74df7a4"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-gjrz"
server_key: "63576fe84517c792262470c99fcff4e0"
server_key_iv: "200785ab36ca9d63a2b49ed958834ecf"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.879675 2458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:02.893199 2458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:02.894827 2458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:02.894950 2458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:02.899499 2458 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:45517
I20260806 22:40:02.899544 2583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:45517 every 8 connection(s)
I20260806 22:40:02.899866 2458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:02.900573 2585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:02.902823 2585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Bootstrap starting.
I20260806 22:40:02.902954 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2458
I20260806 22:40:02.903046 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:02.903208 747 external_mini_cluster.cc:1428] Setting key 497d45c26f3dedb80c0e5ae3b5e5deca
I20260806 22:40:02.903523 2585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:02.903877 2585 log.cc:824] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:02.904543 2585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: No bootstrap required, opened a new log
I20260806 22:40:02.906358 2585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } }
I20260806 22:40:02.906539 2585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:02.906622 2585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118a031da8824527a15e8747b74df7a4, State: Initialized, Role: FOLLOWER
I20260806 22:40:02.906772 2585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } }
I20260806 22:40:02.906849 2585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:02.906888 2585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:02.906932 2585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:02.907505 2585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } }
I20260806 22:40:02.907615 2585 leader_election.cc:302] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 118a031da8824527a15e8747b74df7a4; no voters:
I20260806 22:40:02.907768 2585 leader_election.cc:288] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:02.907806 2593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:02.907979 2593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [term 1 LEADER]: Becoming Leader. State: Replica: 118a031da8824527a15e8747b74df7a4, State: Running, Role: LEADER
I20260806 22:40:02.908036 2585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:02.908154 2593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } }
I20260806 22:40:02.908493 2597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "118a031da8824527a15e8747b74df7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } } }
I20260806 22:40:02.908581 2597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:02.909137 2605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:02.909686 2605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:02.909677 2594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 118a031da8824527a15e8747b74df7a4. Latest consensus state: current_term: 1 leader_uuid: "118a031da8824527a15e8747b74df7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118a031da8824527a15e8747b74df7a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45517 } } }
I20260806 22:40:02.909876 2594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:40:02 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:02.911666 2605 catalog_manager.cc:1347] Generated new cluster ID: b8e303b48a9f47cc9c4eaa4c0a890cb0
I20260806 22:40:02.911720 2605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:02.914173 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:02.903667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45872 (local address 127.0.186.254:45517)
0806 22:40:02.903850 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:02.903859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:02.903885 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:40:02.904414 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:02.904422 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:02.904470 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:02.904601 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:02.905223 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.906901 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.907759 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.908361 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.908468 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:02.911066 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:02.911071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:02.911071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:02.911084 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:02.912709 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.913090 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.913093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.913094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.913179 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.913355 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.913356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.913357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.913601 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:02.913735 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:02.913848 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:40:02.913977 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260806 22:40:02.922619 2605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:02.923347 2605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:02.928565 2605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Generated new TSK 0
I20260806 22:40:02.928730 2605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:02.955252 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:45517
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.036173 2623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.036368 2623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.036391 2623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.037508 2623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.037546 2623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.037567 2623 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:03.037597 2623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:03.039090 2623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:45517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:03.039351 2623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.039552 2623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.044159 2639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.044181 2637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.044219 2634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.044271 2623 server_base.cc:1034] running on GCE node
I20260806 22:40:03.044528 2623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.044765 2623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.045907 2623 hybrid_clock.cc:648] HybridClock initialized: now 1786056003045892 us; error 27 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.048300 2623 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:03.048705 2623 webserver.cc:460] Webserver started at http://127.0.186.193:38551/ using document root <none> and password file <none>
I20260806 22:40:03.048903 2623 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.048950 2623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.049067 2623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:03.050060 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "45d976deb757475ba6e9ae86ba72f4f9"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "4b711f593c10e2ceaddf1554c7330b1b"
server_key_iv: "ce34abe17b6b19db9b18c00b66602944"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.050397 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "45d976deb757475ba6e9ae86ba72f4f9"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "4b711f593c10e2ceaddf1554c7330b1b"
server_key_iv: "ce34abe17b6b19db9b18c00b66602944"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.051648 2623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:03.052342 2649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.052480 2623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:03.052563 2623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "45d976deb757475ba6e9ae86ba72f4f9"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "4b711f593c10e2ceaddf1554c7330b1b"
server_key_iv: "ce34abe17b6b19db9b18c00b66602944"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.052627 2623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.064121 2623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.065779 2623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.065902 2623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.066370 2623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.066416 2623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.066448 2623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.066475 2623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:03.073239 2623 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:44621
I20260806 22:40:03.073323 2833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:44621 every 8 connection(s)
I20260806 22:40:03.073612 2623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:03.075646 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.066863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:36533 (local address 127.0.186.254:45517)
0806 22:40:03.066945 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.066947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.067110 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:40:03.067671 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.067673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.067686 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.067762 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.068323 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.069076 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.070059 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.070366 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.070442 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.073150 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.073151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.073152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.073158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.074566 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.074922 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.074923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.074923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.074969 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.075131 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.075131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.075132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.075226 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.075292 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.075509 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.075579 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:03.075783 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.066807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.193:36533)
0806 22:40:03.066992 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.067032 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.067562 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.067826 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.067830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.068070 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.068246 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.068251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.069175 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.069176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.069993 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.069996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.070473 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.070474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.070588 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.071055 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.071063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.073042 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.074638 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.074642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.074644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.074849 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.075007 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.075007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.075007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.075092 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.075291 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.075296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.075410 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.075489 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.075606 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260806 22:40:03.076272 2834 heartbeater.cc:347] Connected to a master server at 127.0.186.254:45517
I20260806 22:40:03.076355 2834 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.076514 2834 heartbeater.cc:511] Master 127.0.186.254:45517 requested a full tablet report, sending...
I20260806 22:40:03.077080 2511 ts_manager.cc:194] Registered new tserver with Master: 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621)
I20260806 22:40:03.077970 2511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:36533
I20260806 22:40:03.079681 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2623
I20260806 22:40:03.079767 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:03.079910 747 external_mini_cluster.cc:1428] Setting key 615b3573163ac8e487f53f7eed192131
I20260806 22:40:03.082477 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.077358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:52519 (local address 127.0.186.254:45517)
0806 22:40:03.077399 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.077400 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.077527 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:40:03.077578 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.077579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.077589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.077632 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.077955 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.078601 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.079304 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.079495 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.079556 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.080389 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.080390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.080390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.080395 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.081469 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.081779 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.081780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.081780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.081834 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.082001 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.082002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.082002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.082105 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.082176 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.082331 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.082399 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:03.082474 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.077289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.193:52519)
0806 22:40:03.077458 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.077468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.077527 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.077637 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.077637 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.077811 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.077874 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.077875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.078687 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.078687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.079204 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.079205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.079575 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.079576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.079631 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.079994 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.079999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.080320 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.081542 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.081543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.081544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.081716 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.081893 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.081894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.081894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.081951 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.082159 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.082160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.082215 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.082276 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.082372 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:03.104208 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:45517
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.186272 2844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.186520 2844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.186549 2844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.188114 2844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.188171 2844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.188200 2844 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:03.188259 2844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:03.190212 2844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:45517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:03.190434 2844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.190616 2844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.195446 2850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.195468 2852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.195614 2844 server_base.cc:1034] running on GCE node
W20260806 22:40:03.195760 2854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.195945 2844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.196194 2844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.197358 2844 hybrid_clock.cc:648] HybridClock initialized: now 1786056003197346 us; error 29 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.200002 2844 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:03.200415 2844 webserver.cc:460] Webserver started at http://127.0.186.194:37651/ using document root <none> and password file <none>
I20260806 22:40:03.200628 2844 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.200677 2844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.200798 2844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:03.201809 2844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "68e26018d8d74a48827b24f121ab1402"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "fbd10615ccb7ccb8d6f6ef7f19f2796f"
server_key_iv: "cb30fe345af8f204c23834f58fd95747"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.202168 2844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "68e26018d8d74a48827b24f121ab1402"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "fbd10615ccb7ccb8d6f6ef7f19f2796f"
server_key_iv: "cb30fe345af8f204c23834f58fd95747"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.203444 2844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:03.204109 2864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.204257 2844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.204341 2844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "68e26018d8d74a48827b24f121ab1402"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "fbd10615ccb7ccb8d6f6ef7f19f2796f"
server_key_iv: "cb30fe345af8f204c23834f58fd95747"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.204406 2844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.221367 2844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.223213 2844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.223366 2844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.223829 2844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.223868 2844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.223896 2844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.223913 2844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:03.230741 2844 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:42525
I20260806 22:40:03.230814 3056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:42525 every 8 connection(s)
I20260806 22:40:03.231119 2844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:03.233318 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.224278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:54923 (local address 127.0.186.254:45517)
0806 22:40:03.224341 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.224344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.224537 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:40:03.225010 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.225011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.225024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.225096 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.225866 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.226637 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.227901 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.228195 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.228273 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.230816 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.230818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.230818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.230826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.232207 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.232544 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.232545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.232545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.232591 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.232759 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.232760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.232760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.232852 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.232899 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.233197 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.233255 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:40:03.233515 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.224238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.194:54923)
0806 22:40:03.224416 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.224457 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.224914 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.225151 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.225155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.225477 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.225764 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.225771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.226738 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.226740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.227824 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.227828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.228290 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.228291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.228442 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.228942 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.228952 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.230706 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.232274 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.232278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.232279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.232485 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.232630 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.232630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.232631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.232720 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.232965 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.232971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.233086 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.233175 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.233303 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260806 22:40:03.233961 3057 heartbeater.cc:347] Connected to a master server at 127.0.186.254:45517
I20260806 22:40:03.234040 3057 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.234202 3057 heartbeater.cc:511] Master 127.0.186.254:45517 requested a full tablet report, sending...
I20260806 22:40:03.234581 2513 ts_manager.cc:194] Registered new tserver with Master: 68e26018d8d74a48827b24f121ab1402 (127.0.186.194:42525)
I20260806 22:40:03.235002 2513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:54923
I20260806 22:40:03.239038 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2844
I20260806 22:40:03.239127 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:03.239331 747 external_mini_cluster.cc:1428] Setting key d1fb2c3fe69de692fcdcc55533d85345
I20260806 22:40:03.240351 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.234957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:40269 (local address 127.0.186.254:45517)
0806 22:40:03.235000 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.235002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.235088 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:40:03.235171 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.235172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.235184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.235228 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.235779 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236238 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236947 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.237143 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.237194 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.237966 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.237968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.237968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.237975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.239016 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.239394 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.239396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.239396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.239451 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.239804 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.239804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.239804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.239992 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.240054 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.240193 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.240262 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:03.240365 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.234889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.194:40269)
0806 22:40:03.235027 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.235037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.235091 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.235231 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.235232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.235497 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.235702 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.235704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236395 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.236874 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.236875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.237200 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.237201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.237262 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.237619 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.237624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.237885 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.239090 (+ 1205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.239091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.239091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.239313 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.239510 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.239510 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.239511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.239574 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.240056 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.240057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.240104 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.240164 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.240264 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:03.263957 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:45517
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.351423 3070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.351624 3070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.351647 3070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.352799 3070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.352839 3070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.352860 3070 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:03.352891 3070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:03.354391 3070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:45517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:03.354604 3070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.354812 3070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.360363 3081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.360400 3077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.360489 3078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.360728 3070 server_base.cc:1034] running on GCE node
I20260806 22:40:03.360915 3070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.361140 3070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.362275 3070 hybrid_clock.cc:648] HybridClock initialized: now 1786056003362266 us; error 29 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.364683 3070 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:03.365072 3070 webserver.cc:460] Webserver started at http://127.0.186.195:35899/ using document root <none> and password file <none>
I20260806 22:40:03.365289 3070 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.365338 3070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.365460 3070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:03.366449 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "a22e5f8a2b8bc99d0bd29b1807b7ad3f"
server_key_iv: "83850bbd361817764e5ff476a0fc112a"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.366770 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "a22e5f8a2b8bc99d0bd29b1807b7ad3f"
server_key_iv: "83850bbd361817764e5ff476a0fc112a"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.368131 3070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:03.368780 3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.368934 3070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:03.369014 3070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-gjrz"
server_key: "a22e5f8a2b8bc99d0bd29b1807b7ad3f"
server_key_iv: "83850bbd361817764e5ff476a0fc112a"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.369098 3070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.379033 3070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.380726 3070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.380849 3070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.381305 3070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.381345 3070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.381371 3070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.381387 3070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:03.388127 3070 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:44707
I20260806 22:40:03.388520 3070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:03.390880 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.381694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:51733 (local address 127.0.186.254:45517)
0806 22:40:03.381753 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.381756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.381996 (+ 240us) server_negotiation.cc:407] Connection header received
0806 22:40:03.382474 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.382476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.382489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.382578 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.383436 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.384036 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.385460 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.385773 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.385852 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.388298 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.388300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.388300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.388308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.389651 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.390011 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.390013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.390013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.390066 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.390286 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.390287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.390288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.390399 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.390455 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.390711 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.390791 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:40:03.391041 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.381693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.195:51733)
0806 22:40:03.381848 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.381899 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.382383 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.382660 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.382665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.383034 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.383326 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.383333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.384117 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.384118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.385361 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.385365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.385868 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.385869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.386016 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.386470 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.386480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.388175 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.389727 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.389734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.389736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.389945 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.390127 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.390128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.390128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.390230 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.390451 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.390458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.390592 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.390681 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.390829 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260806 22:40:03.391525 3272 heartbeater.cc:347] Connected to a master server at 127.0.186.254:45517
I20260806 22:40:03.391613 3272 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.391791 3272 heartbeater.cc:511] Master 127.0.186.254:45517 requested a full tablet report, sending...
I20260806 22:40:03.392446 2511 ts_manager.cc:194] Registered new tserver with Master: 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707)
I20260806 22:40:03.392885 2511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:51733
I20260806 22:40:03.395857 3271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:44707 every 8 connection(s)
I20260806 22:40:03.398540 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3070
I20260806 22:40:03.398627 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:03.398784 747 external_mini_cluster.cc:1428] Setting key 880475a001a1e3b721f8b1322d9d8715
I20260806 22:40:03.398788 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.392451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.186.195:39047)
0806 22:40:03.393126 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.393137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.393198 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.393369 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.393369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.393614 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.393687 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.393689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.394349 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.394350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.395012 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.395015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.395341 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.395342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.395459 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.396040 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.396044 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.396551 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.397845 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.397846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.397846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.398059 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.398247 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.398248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.398248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.398309 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.398511 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.398512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.398555 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.398620 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.398719 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":79,"threads_started":1}
I20260806 22:40:03.399052 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.393007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:39047 (local address 127.0.186.254:45517)
0806 22:40:03.393052 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.393054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.393199 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:40:03.393242 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.393243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.393253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.393301 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.393760 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.394197 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.395083 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.395255 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.395332 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.396624 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.396625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.396626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.396631 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.397767 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.398125 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.398126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.398126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.398192 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.398360 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.398360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.398360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.398446 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.398497 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.398890 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.398966 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:03.399729 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:03.425940 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.419892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.0.186.254:45517)
0806 22:40:03.419942 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.419944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.419954 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:03.419979 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.419980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.419989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.420031 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.420537 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.421014 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.421702 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.421894 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.421948 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.423893 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.423894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.423894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.423901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.424976 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.425279 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.425280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.425281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.425329 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.425479 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.425479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.425480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.425577 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.425668 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.425776 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.425851 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:03.428006 2511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:03.429631 2511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:03.435815 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.431695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44707 (local address 127.0.0.1:44268)
0806 22:40:03.431967 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.432042 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.432119 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.432360 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.432363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.432364 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.432433 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.432435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.433452 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.433453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.434778 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.434780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435511 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.435605 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435620 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.435683 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.435740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260806 22:40:03.435834 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.431792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.0.186.195:44707)
0806 22:40:03.431948 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.431954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.432206 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:40:03.432252 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.432254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.432271 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.432344 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.432525 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.433373 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.434891 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435437 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435509 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435592 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.435703 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.435740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260806 22:40:03.436110 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.431884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44621 (local address 127.0.0.1:58022)
0806 22:40:03.432083 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.432091 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.432131 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.432811 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.432812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.432815 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.432855 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.432857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.434008 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.434010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.434842 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.434846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435826 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.435895 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435902 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.435949 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.435989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260806 22:40:03.436187 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.432092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.0.186.193:44621)
0806 22:40:03.432467 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.432471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.432480 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:03.432659 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.432662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.432678 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.432747 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.432931 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.433933 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.434969 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435669 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435723 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435788 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.436033 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.436074 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":85,"threads_started":1}
I20260806 22:40:03.436208 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.431820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.0.186.194:42525)
0806 22:40:03.432266 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.432271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.432280 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:03.432315 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.432317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.432330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.432398 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.432573 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.433629 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.434864 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435638 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435709 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435788 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.436069 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.436116 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":103,"threads_started":1}
I20260806 22:40:03.436336 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.431716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42525 (local address 127.0.0.1:52074)
0806 22:40:03.432002 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.432041 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.432122 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.432401 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.432402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.432403 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.432510 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.432512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.433710 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.433711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.434777 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.434780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.435825 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.435826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.435895 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.435904 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.435949 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.435989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":6,"thread_start_us":68,"threads_started":1}
I20260806 22:40:03.436875 2731 tablet_service.cc:1467] Processing CreateTablet for tablet fe17af9cac674dd4b69cdb3070097b4a (DEFAULT_TABLE table=test-table [id=99734ad636374fa1a8606a2cc372ce06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.436815 3173 tablet_service.cc:1467] Processing CreateTablet for tablet fe17af9cac674dd4b69cdb3070097b4a (DEFAULT_TABLE table=test-table [id=99734ad636374fa1a8606a2cc372ce06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.436942 2945 tablet_service.cc:1467] Processing CreateTablet for tablet fe17af9cac674dd4b69cdb3070097b4a (DEFAULT_TABLE table=test-table [id=99734ad636374fa1a8606a2cc372ce06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.437214 3173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe17af9cac674dd4b69cdb3070097b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.437213 2731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe17af9cac674dd4b69cdb3070097b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.437275 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe17af9cac674dd4b69cdb3070097b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.440383 3304 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Bootstrap starting.
I20260806 22:40:03.440817 3305 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Bootstrap starting.
I20260806 22:40:03.441040 3304 tablet_bootstrap.cc:654] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.441061 3306 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Bootstrap starting.
I20260806 22:40:03.441403 3304 log.cc:824] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.441440 3305 tablet_bootstrap.cc:654] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.441695 3306 tablet_bootstrap.cc:654] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.441744 3305 log.cc:824] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.442085 3306 log.cc:824] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.442207 3304 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: No bootstrap required, opened a new log
I20260806 22:40:03.442281 3304 ts_tablet_manager.cc:1397] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:03.442508 3305 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: No bootstrap required, opened a new log
I20260806 22:40:03.442576 3305 ts_tablet_manager.cc:1397] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:03.442775 3306 tablet_bootstrap.cc:492] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: No bootstrap required, opened a new log
I20260806 22:40:03.442835 3306 ts_tablet_manager.cc:1397] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:03.444315 3306 raft_consensus.cc:348] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444446 3306 raft_consensus.cc:374] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.444355 3304 raft_consensus.cc:348] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444489 3306 raft_consensus.cc:729] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d976deb757475ba6e9ae86ba72f4f9, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.444545 3304 raft_consensus.cc:374] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.444494 3305 raft_consensus.cc:348] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444617 3304 raft_consensus.cc:729] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e26018d8d74a48827b24f121ab1402, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.444615 3306 consensus_queue.cc:260] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444645 3305 raft_consensus.cc:374] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.444684 3305 raft_consensus.cc:729] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5fe320b8d748c9a8f3f2bd85b4607a, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.444772 3304 consensus_queue.cc:260] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444801 3305 consensus_queue.cc:260] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.444921 3306 ts_tablet_manager.cc:1428] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:03.444954 2834 heartbeater.cc:503] Master 127.0.186.254:45517 was elected leader, sending a full tablet report...
I20260806 22:40:03.445132 3057 heartbeater.cc:503] Master 127.0.186.254:45517 was elected leader, sending a full tablet report...
I20260806 22:40:03.445353 3305 ts_tablet_manager.cc:1428] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:03.445464 3272 heartbeater.cc:503] Master 127.0.186.254:45517 was elected leader, sending a full tablet report...
I20260806 22:40:03.445560 3304 ts_tablet_manager.cc:1428] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:40:03.475991 3315 raft_consensus.cc:482] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:03.476096 3315 raft_consensus.cc:504] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.476444 3315 leader_election.cc:288] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707), 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621)
I20260806 22:40:03.480046 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.476734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:55145 (local address 127.0.186.195:44707)
0806 22:40:03.476772 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.476773 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.476994 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:40:03.477029 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.477030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.477039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.477118 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.477301 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.477971 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479172 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.479706 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479781 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.479790 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.479947 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.479984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:03.480063 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.476663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:48071 (local address 127.0.186.193:44621)
0806 22:40:03.476713 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.476715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.476724 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:03.476769 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.476770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.476780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.476834 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.477036 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.477785 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479132 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.479719 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479781 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.479790 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.479973 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.480009 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:03.480082 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.476599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44621 (local address 127.0.186.194:48071)
0806 22:40:03.476670 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.476681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.476718 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.476847 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.476848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.476850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.476948 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.476950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.477870 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.477871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.479048 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.479051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.479785 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.479888 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.479890 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.479965 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.479996 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":4}
I20260806 22:40:03.480096 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.476603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44707 (local address 127.0.186.194:55145)
0806 22:40:03.476903 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.476913 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.476984 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.477164 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.477165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.477167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.477232 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.477234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.478043 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.478044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.479098 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.479100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.479786 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.479786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.479879 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.479884 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.479943 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.479985 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":140,"thread_start_us":100,"threads_started":1}
I20260806 22:40:03.480393 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe17af9cac674dd4b69cdb3070097b4a" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d976deb757475ba6e9ae86ba72f4f9" is_pre_election: true
I20260806 22:40:03.480453 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe17af9cac674dd4b69cdb3070097b4a" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" is_pre_election: true
I20260806 22:40:03.480528 2761 raft_consensus.cc:2454] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 0.
I20260806 22:40:03.480582 3195 raft_consensus.cc:2454] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 0.
I20260806 22:40:03.480747 2871 leader_election.cc:302] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45d976deb757475ba6e9ae86ba72f4f9, 68e26018d8d74a48827b24f121ab1402; no voters:
I20260806 22:40:03.480876 3315 raft_consensus.cc:2781] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:03.480939 3315 raft_consensus.cc:482] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:03.480954 3315 raft_consensus.cc:3037] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:40:03.481619 3060 tablet.cc:2369] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:03.481961 3315 raft_consensus.cc:504] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.482074 3315 leader_election.cc:288] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 election: Requested vote from peers 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707), 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621)
I20260806 22:40:03.482224 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe17af9cac674dd4b69cdb3070097b4a" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
I20260806 22:40:03.482235 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe17af9cac674dd4b69cdb3070097b4a" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d976deb757475ba6e9ae86ba72f4f9"
I20260806 22:40:03.482311 3195 raft_consensus.cc:3037] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.482311 2761 raft_consensus.cc:3037] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.483172 3195 raft_consensus.cc:2454] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 1.
I20260806 22:40:03.483189 2761 raft_consensus.cc:2454] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 1.
I20260806 22:40:03.483367 2872 leader_election.cc:302] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e26018d8d74a48827b24f121ab1402, 9d5fe320b8d748c9a8f3f2bd85b4607a; no voters:
I20260806 22:40:03.483508 3315 raft_consensus.cc:2781] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:03.483651 3315 raft_consensus.cc:686] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 1 LEADER]: Becoming Leader. State: Replica: 68e26018d8d74a48827b24f121ab1402, State: Running, Role: LEADER
I20260806 22:40:03.483731 3315 consensus_queue.cc:237] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } }
I20260806 22:40:03.484443 2511 catalog_manager.cc:5495] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e26018d8d74a48827b24f121ab1402 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:03.496929 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.494254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52084 (local address 127.0.186.194:42525)
0806 22:40:03.494294 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.494296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.494304 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:03.494335 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.494336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.494353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.494419 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.494597 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.495127 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.496102 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.496301 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.496355 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.496552 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.496694 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.496771 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.496843 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.496866 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:03.499676 3195 raft_consensus.cc:1261] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Refusing update from remote peer 68e26018d8d74a48827b24f121ab1402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.499972 3318 consensus_queue.cc:1035] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.500401 2761 raft_consensus.cc:1261] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Refusing update from remote peer 68e26018d8d74a48827b24f121ab1402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.500643 3315 consensus_queue.cc:1035] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.502344 3325 mvcc.cc:205] Tried to move back new op lower bound from 7315685390328012800 to 7315685390269882368. Current Snapshot: MvccSnapshot[applied={T|T < 7315685390328012800}]
I20260806 22:40:03.502561 3324 mvcc.cc:205] Tried to move back new op lower bound from 7315685390328012800 to 7315685390269882368. Current Snapshot: MvccSnapshot[applied={T|T < 7315685390328012800 or (T in {7315685390328012800})}]
I20260806 22:40:03.503659 3327 mvcc.cc:205] Tried to move back new op lower bound from 7315685390328012800 to 7315685390269882368. Current Snapshot: MvccSnapshot[applied={T|T < 7315685390328012800}]
I20260806 22:40:03.506213 2511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:03.506340 2511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:03.507138 2511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Sending DeleteTablet for 3 replicas of tablet fe17af9cac674dd4b69cdb3070097b4a
I20260806 22:40:03.507608 3173 tablet_service.cc:1514] Processing DeleteTablet for tablet fe17af9cac674dd4b69cdb3070097b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260806 22:40:03.507864 3335 tablet_replica.cc:330] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: stopping tablet replica
I20260806 22:40:03.507982 3335 raft_consensus.cc:2229] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:03.508086 3335 raft_consensus.cc:2258] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.508065 2511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:03.508373 2511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:03.508647 3335 ts_tablet_manager.cc:1905] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.509470 2731 tablet_service.cc:1514] Processing DeleteTablet for tablet fe17af9cac674dd4b69cdb3070097b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:58022
I20260806 22:40:03.509719 3173 tablet_service.cc:1467] Processing CreateTablet for tablet 56b8da2eb3344f6686f27990170bc530 (DEFAULT_TABLE table=test-table [id=caf9c28bf23a4693894d0a674789926e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.509783 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet fe17af9cac674dd4b69cdb3070097b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:52074
I20260806 22:40:03.509845 3173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b8da2eb3344f6686f27990170bc530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.510004 3337 tablet_replica.cc:330] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: stopping tablet replica
I20260806 22:40:03.510140 3337 raft_consensus.cc:2229] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:03.510304 3337 raft_consensus.cc:2258] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.510818 3335 ts_tablet_manager.cc:1918] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.510893 3335 log.cc:1196] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/fe17af9cac674dd4b69cdb3070097b4a
I20260806 22:40:03.510943 3305 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Bootstrap starting.
I20260806 22:40:03.511149 3335 ts_tablet_manager.cc:1939] T fe17af9cac674dd4b69cdb3070097b4a P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting consensus metadata
I20260806 22:40:03.511369 3305 tablet_bootstrap.cc:654] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.511461 2731 tablet_service.cc:1467] Processing CreateTablet for tablet 56b8da2eb3344f6686f27990170bc530 (DEFAULT_TABLE table=test-table [id=caf9c28bf23a4693894d0a674789926e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.511571 2731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b8da2eb3344f6686f27990170bc530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.511566 2490 catalog_manager.cc:4841] TS 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707): tablet fe17af9cac674dd4b69cdb3070097b4a (table test-table [id=99734ad636374fa1a8606a2cc372ce06]) successfully deleted
I20260806 22:40:03.511792 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 56b8da2eb3344f6686f27990170bc530 (DEFAULT_TABLE table=test-table [id=caf9c28bf23a4693894d0a674789926e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.511811 3337 ts_tablet_manager.cc:1905] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.511911 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b8da2eb3344f6686f27990170bc530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.512005 3336 tablet_replica.cc:330] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: stopping tablet replica
I20260806 22:40:03.512095 3336 raft_consensus.cc:2229] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:03.512187 3336 raft_consensus.cc:2258] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.512593 3336 ts_tablet_manager.cc:1905] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.512869 3306 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Bootstrap starting.
I20260806 22:40:03.513190 3304 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Bootstrap starting.
I20260806 22:40:03.513273 3306 tablet_bootstrap.cc:654] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.513602 3304 tablet_bootstrap.cc:654] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.513864 3337 ts_tablet_manager.cc:1918] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.513953 3337 log.cc:1196] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/fe17af9cac674dd4b69cdb3070097b4a
I20260806 22:40:03.514014 3336 ts_tablet_manager.cc:1918] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.514091 3336 log.cc:1196] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/fe17af9cac674dd4b69cdb3070097b4a
I20260806 22:40:03.514217 3337 ts_tablet_manager.cc:1939] T fe17af9cac674dd4b69cdb3070097b4a P 68e26018d8d74a48827b24f121ab1402: Deleting consensus metadata
I20260806 22:40:03.514338 3336 ts_tablet_manager.cc:1939] T fe17af9cac674dd4b69cdb3070097b4a P 45d976deb757475ba6e9ae86ba72f4f9: Deleting consensus metadata
I20260806 22:40:03.514427 3304 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: No bootstrap required, opened a new log
I20260806 22:40:03.514465 3304 ts_tablet_manager.cc:1397] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:03.514604 2489 catalog_manager.cc:4841] TS 68e26018d8d74a48827b24f121ab1402 (127.0.186.194:42525): tablet fe17af9cac674dd4b69cdb3070097b4a (table test-table [id=99734ad636374fa1a8606a2cc372ce06]) successfully deleted
I20260806 22:40:03.514595 3304 raft_consensus.cc:348] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.514657 3304 raft_consensus.cc:374] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.514674 3304 raft_consensus.cc:729] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e26018d8d74a48827b24f121ab1402, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.514714 3304 consensus_queue.cc:260] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.514765 2489 catalog_manager.cc:4841] TS 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621): tablet fe17af9cac674dd4b69cdb3070097b4a (table test-table [id=99734ad636374fa1a8606a2cc372ce06]) successfully deleted
I20260806 22:40:03.514793 3304 ts_tablet_manager.cc:1428] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.515218 3306 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: No bootstrap required, opened a new log
I20260806 22:40:03.515286 3306 ts_tablet_manager.cc:1397] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:03.515430 3306 raft_consensus.cc:348] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.515482 3306 raft_consensus.cc:374] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.515501 3306 raft_consensus.cc:729] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d976deb757475ba6e9ae86ba72f4f9, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.515539 3306 consensus_queue.cc:260] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.515626 3306 ts_tablet_manager.cc:1428] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.519667 3305 tablet_bootstrap.cc:492] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: No bootstrap required, opened a new log
I20260806 22:40:03.519713 3305 ts_tablet_manager.cc:1397] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260806 22:40:03.519843 3305 raft_consensus.cc:348] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.519899 3305 raft_consensus.cc:374] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.519917 3305 raft_consensus.cc:729] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5fe320b8d748c9a8f3f2bd85b4607a, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.519959 3305 consensus_queue.cc:260] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.520044 3305 ts_tablet_manager.cc:1428] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 22:40:03.574425 2835 tablet.cc:2369] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:03.645984 3273 tablet.cc:2369] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:03.749338 3313 raft_consensus.cc:482] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:03.749444 3313 raft_consensus.cc:504] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.749815 3313 leader_election.cc:288] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e26018d8d74a48827b24f121ab1402 (127.0.186.194:42525), 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707)
I20260806 22:40:03.753679 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.750076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:59867 (local address 127.0.186.195:44707)
0806 22:40:03.750140 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.750142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.750186 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:40:03.750251 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.750252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.750262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.750323 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.750499 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.751240 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.752762 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753304 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753374 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.753384 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.753562 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.753608 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:40:03.753706 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.750035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:56197 (local address 127.0.186.194:42525)
0806 22:40:03.750093 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.750096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.750186 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:40:03.750250 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.750251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.750262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.750323 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.750500 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.751253 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.752763 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753304 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753374 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.753384 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.753614 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.753653 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:03.753705 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.749974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44707 (local address 127.0.186.193:59867)
0806 22:40:03.750135 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.750145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.750194 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.750335 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.750336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.750338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.750431 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.750434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.751339 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.751340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.752676 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.752680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753385 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.753488 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.753493 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.753557 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.753604 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":43,"thread_start_us":63,"threads_started":1}
I20260806 22:40:03.753759 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.749968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42525 (local address 127.0.186.193:56197)
0806 22:40:03.750110 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.750123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.750193 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.750335 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.750336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.750340 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.750443 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.750444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.751342 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.751343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.752687 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.752689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753386 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.753553 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.753557 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.753604 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.753633 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":44,"thread_start_us":73,"threads_started":1}
I20260806 22:40:03.753952 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b8da2eb3344f6686f27990170bc530" candidate_uuid: "45d976deb757475ba6e9ae86ba72f4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" is_pre_election: true
I20260806 22:40:03.754047 3195 raft_consensus.cc:2454] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d976deb757475ba6e9ae86ba72f4f9 in term 0.
I20260806 22:40:03.754020 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b8da2eb3344f6686f27990170bc530" candidate_uuid: "45d976deb757475ba6e9ae86ba72f4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e26018d8d74a48827b24f121ab1402" is_pre_election: true
I20260806 22:40:03.754160 2978 raft_consensus.cc:2454] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d976deb757475ba6e9ae86ba72f4f9 in term 0.
I20260806 22:40:03.754256 2654 leader_election.cc:302] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45d976deb757475ba6e9ae86ba72f4f9, 9d5fe320b8d748c9a8f3f2bd85b4607a; no voters:
I20260806 22:40:03.754442 3313 raft_consensus.cc:2781] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:03.754478 3313 raft_consensus.cc:482] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:03.754491 3313 raft_consensus.cc:3037] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.755205 3313 raft_consensus.cc:504] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.755334 3313 leader_election.cc:288] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [CANDIDATE]: Term 1 election: Requested vote from peers 68e26018d8d74a48827b24f121ab1402 (127.0.186.194:42525), 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707)
I20260806 22:40:03.755518 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b8da2eb3344f6686f27990170bc530" candidate_uuid: "45d976deb757475ba6e9ae86ba72f4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
I20260806 22:40:03.755530 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b8da2eb3344f6686f27990170bc530" candidate_uuid: "45d976deb757475ba6e9ae86ba72f4f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e26018d8d74a48827b24f121ab1402"
I20260806 22:40:03.755594 2978 raft_consensus.cc:3037] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.755594 3195 raft_consensus.cc:3037] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.756457 2978 raft_consensus.cc:2454] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d976deb757475ba6e9ae86ba72f4f9 in term 1.
I20260806 22:40:03.756574 3195 raft_consensus.cc:2454] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d976deb757475ba6e9ae86ba72f4f9 in term 1.
I20260806 22:40:03.756664 2652 leader_election.cc:302] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45d976deb757475ba6e9ae86ba72f4f9, 68e26018d8d74a48827b24f121ab1402; no voters:
I20260806 22:40:03.756769 3313 raft_consensus.cc:2781] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:03.756881 3313 raft_consensus.cc:686] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 LEADER]: Becoming Leader. State: Replica: 45d976deb757475ba6e9ae86ba72f4f9, State: Running, Role: LEADER
I20260806 22:40:03.756949 3313 consensus_queue.cc:237] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.757653 2508 catalog_manager.cc:5495] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "45d976deb757475ba6e9ae86ba72f4f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:40:03 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:03.814944 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.808004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.0.186.254:45517)
0806 22:40:03.808056 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.808059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.808266 (+ 207us) server_negotiation.cc:407] Connection header received
0806 22:40:03.808335 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.808336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.808346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.808406 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.809112 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.809599 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.810257 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.810432 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.810485 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.812701 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.812702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.812703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.812709 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.813932 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.814286 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.814286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.814287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.814335 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.814543 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.814544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.814544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.814634 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.814677 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.814810 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.814878 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":23}
I20260806 22:40:03.815064 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.807966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:45517 (local address 127.0.0.1:45888)
0806 22:40:03.808194 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.808205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.808287 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.808420 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.808421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.808893 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.809029 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.809032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.809667 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.809668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.810194 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.810195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.810539 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.810540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.810595 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.811116 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.811121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.812621 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.814014 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.814023 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.814025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.814225 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.814415 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.814416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.814416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.814484 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.814686 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.814693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.814738 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.814791 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.814944 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":60,"thread_start_us":71,"threads_started":1}
I20260806 22:40:03.817063 2508 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:45888:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:40:03.817319 2508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:03.818765 3173 tablet_service.cc:1467] Processing CreateTablet for tablet f7e936336c7846d5a05444b8624434af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d20d4cf7c2a49ac9261bd9290df5f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:03.818717 2945 tablet_service.cc:1467] Processing CreateTablet for tablet f7e936336c7846d5a05444b8624434af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d20d4cf7c2a49ac9261bd9290df5f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:03.818904 3173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e936336c7846d5a05444b8624434af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.818919 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e936336c7846d5a05444b8624434af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.820286 3304 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: Bootstrap starting.
I20260806 22:40:03.818707 2731 tablet_service.cc:1467] Processing CreateTablet for tablet f7e936336c7846d5a05444b8624434af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d20d4cf7c2a49ac9261bd9290df5f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:03.820693 3304 tablet_bootstrap.cc:654] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.820708 2731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e936336c7846d5a05444b8624434af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.821250 3304 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: No bootstrap required, opened a new log
I20260806 22:40:03.821300 3304 ts_tablet_manager.cc:1397] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:03.821400 3304 raft_consensus.cc:348] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.821465 3304 raft_consensus.cc:374] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.821481 3304 raft_consensus.cc:729] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e26018d8d74a48827b24f121ab1402, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.821507 3304 consensus_queue.cc:260] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.821596 3304 ts_tablet_manager.cc:1428] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:03.821627 3332 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.821719 3332 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.822161 3306 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: Bootstrap starting.
I20260806 22:40:03.822190 3305 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: Bootstrap starting.
I20260806 22:40:03.822631 3305 tablet_bootstrap.cc:654] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.822721 3306 tablet_bootstrap.cc:654] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.823470 3305 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: No bootstrap required, opened a new log
I20260806 22:40:03.823534 3305 ts_tablet_manager.cc:1397] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:03.823674 3305 raft_consensus.cc:348] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.823721 3305 raft_consensus.cc:374] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.823733 3305 raft_consensus.cc:729] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5fe320b8d748c9a8f3f2bd85b4607a, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.823764 3305 consensus_queue.cc:260] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.823836 3305 ts_tablet_manager.cc:1428] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.823993 3316 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.824110 3316 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.824565 3306 tablet_bootstrap.cc:492] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: No bootstrap required, opened a new log
I20260806 22:40:03.824618 3306 ts_tablet_manager.cc:1397] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:40:03.824905 3306 raft_consensus.cc:348] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.824978 3306 raft_consensus.cc:374] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.824996 3306 raft_consensus.cc:729] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d976deb757475ba6e9ae86ba72f4f9, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.825033 3306 consensus_queue.cc:260] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.825119 3306 ts_tablet_manager.cc:1428] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.825186 3313 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.825271 3313 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.943732 3332 raft_consensus.cc:482] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:03.943827 3332 raft_consensus.cc:504] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.944008 3332 leader_election.cc:288] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621), 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707)
I20260806 22:40:03.944198 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e936336c7846d5a05444b8624434af" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d976deb757475ba6e9ae86ba72f4f9" is_pre_election: true
I20260806 22:40:03.944212 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e936336c7846d5a05444b8624434af" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" is_pre_election: true
I20260806 22:40:03.944298 3195 raft_consensus.cc:2454] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 0.
I20260806 22:40:03.944301 2761 raft_consensus.cc:2454] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 0.
I20260806 22:40:03.944455 2871 leader_election.cc:302] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45d976deb757475ba6e9ae86ba72f4f9, 68e26018d8d74a48827b24f121ab1402; no voters:
I20260806 22:40:03.944574 3332 raft_consensus.cc:2781] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:03.944610 3332 raft_consensus.cc:482] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:03.944630 3332 raft_consensus.cc:3037] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.945472 3332 raft_consensus.cc:504] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.945601 3332 leader_election.cc:288] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 election: Requested vote from peers 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621), 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707)
I20260806 22:40:03.945796 3195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e936336c7846d5a05444b8624434af" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a"
I20260806 22:40:03.945886 3195 raft_consensus.cc:3037] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.945910 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e936336c7846d5a05444b8624434af" candidate_uuid: "68e26018d8d74a48827b24f121ab1402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d976deb757475ba6e9ae86ba72f4f9"
I20260806 22:40:03.945983 2761 raft_consensus.cc:3037] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.946642 3195 raft_consensus.cc:2454] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 1.
I20260806 22:40:03.946838 2872 leader_election.cc:302] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e26018d8d74a48827b24f121ab1402, 9d5fe320b8d748c9a8f3f2bd85b4607a; no voters:
I20260806 22:40:03.946978 2761 raft_consensus.cc:2454] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e26018d8d74a48827b24f121ab1402 in term 1.
I20260806 22:40:03.947019 3332 raft_consensus.cc:2781] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:03.947091 3332 raft_consensus.cc:686] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [term 1 LEADER]: Becoming Leader. State: Replica: 68e26018d8d74a48827b24f121ab1402, State: Running, Role: LEADER
I20260806 22:40:03.947147 3332 consensus_queue.cc:237] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } }
I20260806 22:40:03.947448 3315 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: TxnStatusTablet state changed. Reason: New leader 68e26018d8d74a48827b24f121ab1402. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.947537 3315 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:03.947726 3366 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:40:03.947753 2508 catalog_manager.cc:5495] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e26018d8d74a48827b24f121ab1402 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:03.947867 3366 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:40:03.971778 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.969227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.0.186.194:42525)
0806 22:40:03.969289 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.969291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.969306 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:40:03.969393 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.969394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.969405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.969467 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.969590 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.970323 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.971087 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.971331 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.971397 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.971582 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.971613 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.971647 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.971706 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.971723 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:40:03.971833 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.969153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42525 (local address 127.0.0.1:52100)
0806 22:40:03.969261 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.969274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.969326 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.969484 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.969485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.969486 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:03.969535 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.969537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.970403 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.970404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.970989 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.970991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.971404 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.971405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.971482 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.971503 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.971651 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.971656 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.971702 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.971751 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:40:03.972880 2761 raft_consensus.cc:1261] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Refusing update from remote peer 68e26018d8d74a48827b24f121ab1402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.972880 3195 raft_consensus.cc:1261] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Refusing update from remote peer 68e26018d8d74a48827b24f121ab1402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.973106 3315 consensus_queue.cc:1035] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.973227 3332 consensus_queue.cc:1035] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.973688 3316 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: TxnStatusTablet state changed. Reason: New leader 68e26018d8d74a48827b24f121ab1402. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.973783 3316 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.973841 3313 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: TxnStatusTablet state changed. Reason: New leader 68e26018d8d74a48827b24f121ab1402. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.973914 3313 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.974522 3315 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.974524 3332 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.974623 3315 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:03.974630 3332 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 68e26018d8d74a48827b24f121ab1402: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:03.974988 3313 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.975065 3313 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 45d976deb757475ba6e9ae86ba72f4f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.975884 3316 tablet_replica.cc:439] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68e26018d8d74a48827b24f121ab1402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d976deb757475ba6e9ae86ba72f4f9" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44621 } } peers { permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 } } peers { permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 } } }
I20260806 22:40:03.975975 3316 tablet_replica.cc:442] T f7e936336c7846d5a05444b8624434af P 9d5fe320b8d748c9a8f3f2bd85b4607a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:03.979233 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.976768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.0.186.193:44621)
0806 22:40:03.976813 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.976815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.976824 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:03.976893 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.976894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.976913 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.976987 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.977075 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.977769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.978459 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.978765 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.978830 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.978917 (+ 87us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.979046 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.979108 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.979142 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.979162 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:03.984091 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.981666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:34695 (local address 127.0.186.194:42525)
0806 22:40:03.981708 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.981710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.981830 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:40:03.981886 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.981887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.981897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.981948 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.982157 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.982653 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.983467 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.983656 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.983708 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.983874 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.983906 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.983950 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.984007 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.984027 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:03.984134 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.981609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42525 (local address 127.0.186.193:34695)
0806 22:40:03.981772 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.981781 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.981830 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.981952 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.981952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.981985 (+ 33us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:03.982063 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.982064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.982727 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.982728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.983393 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.983395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.983733 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.983734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.983794 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.983807 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.983965 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.983967 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.984006 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.984046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260806 22:40:03.986109 3195 raft_consensus.cc:1261] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Refusing update from remote peer 45d976deb757475ba6e9ae86ba72f4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.986192 2978 raft_consensus.cc:1261] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Refusing update from remote peer 45d976deb757475ba6e9ae86ba72f4f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.986335 3313 consensus_queue.cc:1035] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d5fe320b8d748c9a8f3f2bd85b4607a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.986474 3313 consensus_queue.cc:1035] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e26018d8d74a48827b24f121ab1402" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.995561 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.993171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:36489 (local address 127.0.186.193:44621)
0806 22:40:03.993206 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.993207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.993252 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:40:03.993293 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.993294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.993303 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.993352 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.993461 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.993940 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.994830 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.995096 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.995151 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.995333 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.995365 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.995416 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.995482 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.995500 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:03.995601 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.993035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44621 (local address 127.0.186.194:36489)
0806 22:40:03.993184 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.993195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.993249 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.993356 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.993357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.993358 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:03.993409 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.993410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.994034 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.994035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.994732 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.994735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.995166 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.995166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.995223 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.995238 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.995422 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.995424 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.995478 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.995523 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260806 22:40:04.004798 2508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:04.004896 2508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:04.005486 2508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 118a031da8824527a15e8747b74df7a4: Sending DeleteTablet for 3 replicas of tablet 56b8da2eb3344f6686f27990170bc530
I20260806 22:40:04.005865 3173 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b8da2eb3344f6686f27990170bc530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260806 22:40:04.005949 3335 tablet_replica.cc:330] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: stopping tablet replica
I20260806 22:40:04.005985 3335 raft_consensus.cc:2229] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:04.006018 3335 raft_consensus.cc:2258] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.006135 2731 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b8da2eb3344f6686f27990170bc530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:58022
I20260806 22:40:04.006206 3336 tablet_replica.cc:330] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: stopping tablet replica
I20260806 22:40:04.006242 3336 raft_consensus.cc:2229] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:04.006317 3336 raft_consensus.cc:2258] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.006454 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b8da2eb3344f6686f27990170bc530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:52074
I20260806 22:40:04.006554 3337 tablet_replica.cc:330] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: stopping tablet replica
I20260806 22:40:04.006629 3337 raft_consensus.cc:2229] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:04.006666 3337 raft_consensus.cc:2258] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.006760 3335 ts_tablet_manager.cc:1905] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:04.007012 3337 ts_tablet_manager.cc:1905] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:04.007337 3336 ts_tablet_manager.cc:1905] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:04.008724 3337 ts_tablet_manager.cc:1918] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:04.008776 3337 log.cc:1196] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/56b8da2eb3344f6686f27990170bc530
I20260806 22:40:04.008999 3337 ts_tablet_manager.cc:1939] T 56b8da2eb3344f6686f27990170bc530 P 68e26018d8d74a48827b24f121ab1402: Deleting consensus metadata
I20260806 22:40:04.010447 3335 ts_tablet_manager.cc:1918] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:04.010509 3335 log.cc:1196] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/56b8da2eb3344f6686f27990170bc530
I20260806 22:40:04.010686 3335 ts_tablet_manager.cc:1939] T 56b8da2eb3344f6686f27990170bc530 P 9d5fe320b8d748c9a8f3f2bd85b4607a: Deleting consensus metadata
I20260806 22:40:04.010872 3336 ts_tablet_manager.cc:1918] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:04.010895 2489 catalog_manager.cc:4841] TS 68e26018d8d74a48827b24f121ab1402 (127.0.186.194:42525): tablet 56b8da2eb3344f6686f27990170bc530 (table test-table [id=caf9c28bf23a4693894d0a674789926e]) successfully deleted
I20260806 22:40:04.010933 3336 log.cc:1196] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/56b8da2eb3344f6686f27990170bc530
I20260806 22:40:04.011118 3336 ts_tablet_manager.cc:1939] T 56b8da2eb3344f6686f27990170bc530 P 45d976deb757475ba6e9ae86ba72f4f9: Deleting consensus metadata
I20260806 22:40:04.011425 2489 catalog_manager.cc:4841] TS 45d976deb757475ba6e9ae86ba72f4f9 (127.0.186.193:44621): tablet 56b8da2eb3344f6686f27990170bc530 (table test-table [id=caf9c28bf23a4693894d0a674789926e]) successfully deleted
I20260806 22:40:04.011451 2490 catalog_manager.cc:4841] TS 9d5fe320b8d748c9a8f3f2bd85b4607a (127.0.186.195:44707): tablet 56b8da2eb3344f6686f27990170bc530 (table test-table [id=caf9c28bf23a4693894d0a674789926e]) successfully deleted
Aug 06 22:40:04 dist-test-slave-gjrz krb5kdc[2438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:04.021929 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.011760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.0.186.193:44621)
0806 22:40:04.011801 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.011803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.011936 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:40:04.012009 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.012011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.012022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.012075 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:04.012820 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.013548 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.014632 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.014933 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.015001 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.018489 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:04.018498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:04.018499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:04.018507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:04.020495 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.020896 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.020900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.020901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.020991 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.021190 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.021191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.021192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.021374 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:04.021524 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.021604 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.021687 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:40:04.022239 2825 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58038
I20260806 22:40:04.030423 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.023327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.0.186.254:45517)
0806 22:40:04.023371 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.023373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.023382 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:04.023432 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.023433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.023444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.023503 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:04.024078 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.024845 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.025851 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.026133 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.026211 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.027604 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:04.027606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:04.027606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:04.027615 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:04.029248 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.029630 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.029632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.029632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.029697 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.029894 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.029895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.029895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.030033 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:04.030139 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.030245 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.030327 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:40:04.030642 2508 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45892
I20260806 22:40:04.031490 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2623
I20260806 22:40:04.041198 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2844
I20260806 22:40:04.053516 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3070
I20260806 22:40:04.066645 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2458
2026-08-06T22:40:04Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2563 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:04 dist-test-slave-gjrz krb5kdc[3397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:04 dist-test-slave-gjrz krb5kdc[3397](info): set up 2 sockets
Aug 06 22:40:04 dist-test-slave-gjrz krb5kdc[3397](info): commencing operation
krb5kdc: starting...
W20260806 22:40:05.219549 747 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:05.335907 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37341
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:44465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:05.416173 3417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:05.416368 3417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:05.416396 3417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:05.417529 3417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:05.417569 3417 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:40:05.417588 3417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:05.417604 3417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:05.417621 3417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:05.417640 3417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:05.419118 3417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:44465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37341
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:05.419363 3417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:05.419571 3417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:05.424008 3427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.424017 3426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.424068 3432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.424335 3417 server_base.cc:1034] running on GCE node
I20260806 22:40:05.424549 3417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:05.424790 3417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:05.425922 3417 hybrid_clock.cc:648] HybridClock initialized: now 1786056005425911 us; error 32 us; skew 500 ppm
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:05.428373 3417 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:05.428771 3417 webserver.cc:460] Webserver started at http://127.0.186.254:43059/ using document root <none> and password file <none>
I20260806 22:40:05.428980 3417 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:05.429028 3417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:05.429157 3417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:05.430164 3417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "18d3a384b5494cbaa15e38d751574428"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4514eb680205e5063c5b28a663f009a9"
server_key_iv: "cd49edc35bea971df173626c48dfe467"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.430524 3417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "18d3a384b5494cbaa15e38d751574428"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4514eb680205e5063c5b28a663f009a9"
server_key_iv: "cd49edc35bea971df173626c48dfe467"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.431864 3417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:40:05.432550 3447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.432698 3417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:05.432780 3417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "18d3a384b5494cbaa15e38d751574428"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4514eb680205e5063c5b28a663f009a9"
server_key_iv: "cd49edc35bea971df173626c48dfe467"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.432839 3417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:05.454205 3417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:05.456028 3417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:05.456171 3417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:05.460872 3417 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37341
I20260806 22:40:05.460963 3550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37341 every 8 connection(s)
I20260806 22:40:05.461243 3417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:05.461920 3551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:05.464148 3551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Bootstrap starting.
I20260806 22:40:05.464689 3551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:05.464926 3551 log.cc:824] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:05.465638 3551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: No bootstrap required, opened a new log
I20260806 22:40:05.467931 3551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } }
I20260806 22:40:05.468073 3551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:05.468094 3551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d3a384b5494cbaa15e38d751574428, State: Initialized, Role: FOLLOWER
I20260806 22:40:05.468205 3551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } }
I20260806 22:40:05.468281 3551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:05.468309 3551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:05.468341 3551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:05.468819 3551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } }
I20260806 22:40:05.468905 3551 leader_election.cc:302] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18d3a384b5494cbaa15e38d751574428; no voters:
I20260806 22:40:05.469040 3551 leader_election.cc:288] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:05.469099 3556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:05.469250 3551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:05.469270 3556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [term 1 LEADER]: Becoming Leader. State: Replica: 18d3a384b5494cbaa15e38d751574428, State: Running, Role: LEADER
I20260806 22:40:05.469439 3556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } }
I20260806 22:40:05.469805 3557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18d3a384b5494cbaa15e38d751574428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } } }
I20260806 22:40:05.469887 3557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:05.470069 3566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:05.470077 3558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18d3a384b5494cbaa15e38d751574428. Latest consensus state: current_term: 1 leader_uuid: "18d3a384b5494cbaa15e38d751574428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18d3a384b5494cbaa15e38d751574428" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37341 } } }
I20260806 22:40:05.470567 3558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:05.470664 3566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:05.470800 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3417
I20260806 22:40:05.470893 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:05.471065 747 external_mini_cluster.cc:1428] Setting key 6f3ec142282fcf2c1671028c49da2383
I20260806 22:40:05.472186 3566 catalog_manager.cc:1347] Generated new cluster ID: b0f5d2f0a12f4558bf5bab30192af7b9
I20260806 22:40:05.472288 3566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:05.484655 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.471486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.0.186.254:37341)
0806 22:40:05.475397 (+ 3911us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.475405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.475432 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:40:05.475536 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.475541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.475582 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.475709 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.476321 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.477697 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.478443 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.478875 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.478969 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.481153 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.481158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.481158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.481170 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.482990 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.483346 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.483350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.483350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.483434 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.483617 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.483617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.483618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.483838 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.484100 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.484175 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.484294 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3863,"thread_start_us":60,"threads_started":1}
I20260806 22:40:05.496181 3566 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:05.496891 3566 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:05.503775 3566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Generated new TSK 0
I20260806 22:40:05.503945 3566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:05.531996 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37341
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:44465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:05.629933 3588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:05.630142 3588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:05.630167 3588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:05.631245 3588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:05.631309 3588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:05.631335 3588 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:40:05.631347 3588 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:05.631378 3588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:05.632831 3588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:44465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:05.633059 3588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:05.633256 3588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:05.637972 3595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.638015 3594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.638175 3588 server_base.cc:1034] running on GCE node
W20260806 22:40:05.638275 3597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.638484 3588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:05.638715 3588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:05.639883 3588 hybrid_clock.cc:648] HybridClock initialized: now 1786056005639867 us; error 25 us; skew 500 ppm
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:05.642347 3588 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:05.642755 3588 webserver.cc:460] Webserver started at http://127.0.186.193:38635/ using document root <none> and password file <none>
I20260806 22:40:05.642953 3588 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:05.643011 3588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:05.643138 3588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:05.644183 3588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "9c42946b6e3c4cacadb3a4ad8489ff04"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "c8538ee569b59ab70c7118e5f95ab17d"
server_key_iv: "15a2612f9e3f42188e3bd376106d7a5a"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.644516 3588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "9c42946b6e3c4cacadb3a4ad8489ff04"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "c8538ee569b59ab70c7118e5f95ab17d"
server_key_iv: "15a2612f9e3f42188e3bd376106d7a5a"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.645690 3588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:05.646389 3612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.646559 3588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:05.646646 3588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "9c42946b6e3c4cacadb3a4ad8489ff04"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "c8538ee569b59ab70c7118e5f95ab17d"
server_key_iv: "15a2612f9e3f42188e3bd376106d7a5a"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.646710 3588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:05.660113 3588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:05.661815 3588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:05.661959 3588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:05.662472 3588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:05.662521 3588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.662549 3588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:05.662566 3588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:05.669380 3588 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:37865
I20260806 22:40:05.669456 3802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:37865 every 8 connection(s)
I20260806 22:40:05.669788 3588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:05.671595 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.662832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54545 (local address 127.0.186.254:37341)
0806 22:40:05.662895 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.662898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.663124 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:40:05.663816 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.663818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.663830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.663862 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.664651 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.665158 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.666113 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.666316 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.666371 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.668893 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.668894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.668895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.668903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.670437 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.670791 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.670792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.670792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.670845 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.671048 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.671048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.671049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.671153 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.671203 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.671451 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.671521 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:40:05.671725 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.662802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.193:54545)
0806 22:40:05.662970 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.663016 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:05.663714 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:05.663946 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:05.663950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:05.664281 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:05.664566 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.664573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.665253 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.665254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.666031 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.666034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.666391 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.666391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.666512 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.666925 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:05.666933 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:05.668795 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:05.670528 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.670531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.670533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.670731 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.670886 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.670887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.670887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.670997 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.671200 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:05.671203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:05.671348 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 22:40:05.671428 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:40:05.671551 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 22:40:05.672206 3804 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37341
I20260806 22:40:05.672286 3804 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:05.672475 3804 heartbeater.cc:511] Master 127.0.186.254:37341 requested a full tablet report, sending...
I20260806 22:40:05.672995 3473 ts_manager.cc:194] Registered new tserver with Master: 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865)
I20260806 22:40:05.673765 3473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:54545
I20260806 22:40:05.677021 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3588
I20260806 22:40:05.677107 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:05.677277 747 external_mini_cluster.cc:1428] Setting key e279a4cf439fb09d265b32cfd3709b57
I20260806 22:40:05.677896 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.673133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.193:43839)
0806 22:40:05.673263 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.673273 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:05.673322 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:05.673431 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:05.673431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:05.673577 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:05.673621 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.673622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.674365 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.674366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.674900 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.674902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.675181 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.675181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.675234 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.675604 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:05.675608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:05.675853 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:05.676963 (+ 1110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.676964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.676964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.677139 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.677329 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.677329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.677330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.677381 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.677608 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:05.677609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:05.677652 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:40:05.677708 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:40:05.677803 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260806 22:40:05.677893 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.673168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:43839 (local address 127.0.186.254:37341)
0806 22:40:05.673208 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.673209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.673328 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:40:05.673367 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.673368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.673377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.673427 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.673671 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.674272 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.674960 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.675134 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.675174 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.675910 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.675911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.675911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.675915 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.676897 (+ 982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.677219 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.677220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.677220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.677278 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.677430 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.677431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.677432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.677548 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.677604 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.677736 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.677807 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:05.701450 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37341
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:44465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:05.780652 3818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:05.780841 3818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:05.780864 3818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:05.781980 3818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:05.782019 3818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:05.782038 3818 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:40:05.782061 3818 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:05.782091 3818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:05.783545 3818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:44465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:05.783757 3818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:05.783962 3818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:05.788599 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.788611 3837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.788645 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.788779 3818 server_base.cc:1034] running on GCE node
I20260806 22:40:05.789098 3818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:05.789314 3818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:05.790459 3818 hybrid_clock.cc:648] HybridClock initialized: now 1786056005790448 us; error 26 us; skew 500 ppm
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:05.792861 3818 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:05.793269 3818 webserver.cc:460] Webserver started at http://127.0.186.194:35485/ using document root <none> and password file <none>
I20260806 22:40:05.793505 3818 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:05.793551 3818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:05.793678 3818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:05.794801 3818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "e7c901f3972249fea2d0d194eb4777c4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4f0be276817c28a0ecdd026552ab5603"
server_key_iv: "fc82a2210ba7f833a205e7181ab35c97"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.795171 3818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "e7c901f3972249fea2d0d194eb4777c4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4f0be276817c28a0ecdd026552ab5603"
server_key_iv: "fc82a2210ba7f833a205e7181ab35c97"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.796492 3818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:05.797119 3848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.797266 3818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:05.797334 3818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "e7c901f3972249fea2d0d194eb4777c4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "4f0be276817c28a0ecdd026552ab5603"
server_key_iv: "fc82a2210ba7f833a205e7181ab35c97"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.797402 3818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:05.832345 3818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:05.834399 3818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:05.834535 3818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:05.835101 3818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:05.835140 3818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.835161 3818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:05.835172 3818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:05.843225 3818 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:39139
I20260806 22:40:05.843323 4033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:39139 every 8 connection(s)
I20260806 22:40:05.843633 3818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:05.845110 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.835530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:60487 (local address 127.0.186.254:37341)
0806 22:40:05.835593 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.835595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.835779 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:40:05.836294 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.836296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.836311 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.836387 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.837079 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.837829 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.839348 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.839651 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.839729 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.842606 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.842608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.842608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.842616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.844110 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.844429 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.844430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.844430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.844479 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.844632 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.844632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.844632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.844728 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.844772 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.844972 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.845031 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:40:05.845238 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.835485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.194:60487)
0806 22:40:05.835659 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.835700 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:05.836188 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:05.836458 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:05.836463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:05.836782 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:05.836973 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.836978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.837926 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.837927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.839229 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.839233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.839735 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.839735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.839866 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.840352 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:05.840362 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:05.842499 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:05.844167 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.844169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.844171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.844369 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.844512 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.844513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.844513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.844593 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.844764 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:05.844767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:05.844880 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 22:40:05.844957 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:40:05.845069 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260806 22:40:05.845678 4035 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37341
I20260806 22:40:05.845753 4035 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:05.845916 4035 heartbeater.cc:511] Master 127.0.186.254:37341 requested a full tablet report, sending...
I20260806 22:40:05.846163 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3818
I20260806 22:40:05.846258 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:05.846441 747 external_mini_cluster.cc:1428] Setting key 6521c85cab56028ac6f7284f78817c29
I20260806 22:40:05.846644 3474 ts_manager.cc:194] Registered new tserver with Master: e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139)
I20260806 22:40:05.847095 3474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:60487
I20260806 22:40:05.852294 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.847190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:54345 (local address 127.0.186.254:37341)
0806 22:40:05.847244 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.847246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.847253 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:05.847290 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.847290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.847300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.847353 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.847635 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.848085 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.848888 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.849134 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.849188 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.849999 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.850001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.850001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.850007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.851249 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.851631 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.851632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.851632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.851680 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.851852 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.851852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.851852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.851974 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.852024 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.852178 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.852236 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:40:05.852365 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.846654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.194:54345)
0806 22:40:05.846778 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.846787 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:05.846831 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:05.847361 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:05.847362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:05.847519 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:05.847568 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.847569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.848169 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.848170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.848798 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.848801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.849202 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.849203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.849282 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.849641 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:05.849646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:05.849908 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:05.851348 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.851349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.851350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.851555 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.851735 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.851736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.851736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.851799 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.852034 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:05.852036 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:05.852081 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:05.852163 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:40:05.852259 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:05.872205 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37341
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:44465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:05.954347 4053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:05.954537 4053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:05.954561 4053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:05.955675 4053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:05.955714 4053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:05.955734 4053 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:40:05.955744 4053 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:05.955760 4053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:05.957170 4053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:44465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:05.957370 4053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:05.957569 4053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:05.962109 4059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:05.962116 4062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.962132 4053 server_base.cc:1034] running on GCE node
W20260806 22:40:05.962121 4060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:05.962534 4053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:05.962764 4053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:05.963893 4053 hybrid_clock.cc:648] HybridClock initialized: now 1786056005963883 us; error 29 us; skew 500 ppm
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:05.966162 4053 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:05.966598 4053 webserver.cc:460] Webserver started at http://127.0.186.195:40041/ using document root <none> and password file <none>
I20260806 22:40:05.966789 4053 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:05.966835 4053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:05.966953 4053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:05.968006 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "b5da924d8b0344afb1b27013cd2aace4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "e5052ef5f7ea08a27bdc3af427361c39"
server_key_iv: "45f0b54653be3f1902c670af543aef13"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.968338 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "b5da924d8b0344afb1b27013cd2aace4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "e5052ef5f7ea08a27bdc3af427361c39"
server_key_iv: "45f0b54653be3f1902c670af543aef13"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.969650 4053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:05.970391 4069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.970638 4053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:05.970721 4053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "b5da924d8b0344afb1b27013cd2aace4"
format_stamp: "Formatted at 2026-08-06 22:40:05 on dist-test-slave-gjrz"
server_key: "e5052ef5f7ea08a27bdc3af427361c39"
server_key_iv: "45f0b54653be3f1902c670af543aef13"
server_key_version: "encryptionkey@0"
I20260806 22:40:05.970786 4053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:05.986630 4053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:05.988597 4053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:05.988730 4053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:05.989243 4053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:05.989285 4053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:05.989313 4053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:05.989341 4053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:05 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:05.996085 4053 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:36017
I20260806 22:40:05.996157 4246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:36017 every 8 connection(s)
I20260806 22:40:05.996435 4053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:05.996861 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 4053
I20260806 22:40:05.996954 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:05.997133 747 external_mini_cluster.cc:1428] Setting key cf2f04dfddc0228851f610de0d1c3613
I20260806 22:40:05.998704 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.989606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:48205 (local address 127.0.186.254:37341)
0806 22:40:05.989668 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:05.989670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:05.989881 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:40:05.990361 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:05.990362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:05.990376 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:05.990464 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:05.991065 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.991844 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.992834 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.993041 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.993097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.996051 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:05.996052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:05.996053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:05.996060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:05.997574 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.998017 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.998018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.998019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.998077 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:05.998255 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:05.998256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:05.998256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:05.998350 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:05.998392 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:05.998582 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:40:05.998641 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:05.998880 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.989585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.195:48205)
0806 22:40:05.989761 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.989801 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:05.990270 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:05.990543 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:05.990546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:05.990802 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:05.990989 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.990994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.991938 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.991939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.992744 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:05.992746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:05.993112 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:05.993114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:05.993237 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:05.993712 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:05.993722 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:05.995949 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:05.997649 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.997655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.997658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.997949 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.998119 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:05.998119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:05.998120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:05.998215 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:05.998384 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:05.998386 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:05.998494 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:40:05.998567 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:40:05.998684 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260806 22:40:05.999303 4247 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37341
I20260806 22:40:05.999382 4247 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:05.999560 4247 heartbeater.cc:511] Master 127.0.186.254:37341 requested a full tablet report, sending...
I20260806 22:40:05.999892 3473 ts_manager.cc:194] Registered new tserver with Master: b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:06.000312 3473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:48205
I20260806 22:40:06.005267 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.000358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:48407 (local address 127.0.186.254:37341)
0806 22:40:06.000462 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.000463 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.000472 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:06.000528 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.000528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.000536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.000578 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.000884 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.001357 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.002036 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.002214 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.002265 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.003038 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.003039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.003040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:06.003046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.004322 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.004613 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.004614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.004614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.004664 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.004822 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.004823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.004823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.004927 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.004980 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.005111 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.005178 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:40:06.005275 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.000293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.186.195:48407)
0806 22:40:06.000425 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.000436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.000480 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.000584 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.000585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.000760 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:06.000809 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.000811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.001444 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.001445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.001963 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.001965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.002270 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.002271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.002344 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.002691 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:06.002696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:06.002959 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:06.004397 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.004398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.004398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.004553 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.004716 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.004717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.004717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.004773 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.004980 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:06.004980 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:06.005023 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.005083 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.005179 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260806 22:40:06.008656 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:40:06 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:40:06 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:06.031159 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.024773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.0.186.254:37341)
0806 22:40:06.024818 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.024819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.024829 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:06.024896 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.024897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.024907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.024965 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.025586 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.026072 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.026738 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.026928 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.026982 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.029092 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.029093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.029094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:06.029100 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.030348 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.030618 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.030619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.030619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.030663 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.030781 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.030782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.030782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.030896 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.030968 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.031033 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.031092 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:06.032637 3473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:06.034202 3473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:06.040716 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:37865 (local address 127.0.0.1:40974)
0806 22:40:06.036711 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.036750 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.036816 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.037424 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.037425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.037427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.037475 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.037477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.038417 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.038418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.039714 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.039717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.040420 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.040421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.040501 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.040516 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.040583 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.040640 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":60,"threads_started":1}
I20260806 22:40:06.040899 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.0.186.194:39139)
0806 22:40:06.036991 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.036997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.037026 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:40:06.037108 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.037111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.037127 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.037390 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.037463 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.038560 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.039804 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.040521 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.040591 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.040645 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.040767 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.040807 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260806 22:40:06.040856 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:39139 (local address 127.0.0.1:50916)
0806 22:40:06.036967 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.036978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.037031 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.037239 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.037242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.037245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.037314 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.037316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.038653 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.038654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.039718 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.039720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.040608 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.040609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.040683 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.040687 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.040743 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.040785 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260806 22:40:06.041301 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.0.186.193:37865)
0806 22:40:06.037217 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.037223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.037232 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:06.037303 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.037305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.037321 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.037533 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.037577 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.038336 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.039829 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.040319 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.040417 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.040481 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.040929 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.040970 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":210,"threads_started":1}
I20260806 22:40:06.041424 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:36017 (local address 127.0.0.1:38172)
0806 22:40:06.036998 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.037009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.037061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.037326 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.037327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.037329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.037370 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.037371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.039354 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.039355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.040282 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.040286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.041218 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.041219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.041278 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.041282 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.041328 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.041363 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":95,"threads_started":1}
I20260806 22:40:06.041690 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.036793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.0.186.195:36017)
0806 22:40:06.037139 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.037142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.037154 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:06.037187 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.037188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.037202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.037268 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.037441 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.038546 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.040377 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.041045 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.041126 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.041184 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.041562 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.041604 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":78,"threads_started":1}
I20260806 22:40:06.041810 3930 tablet_service.cc:1467] Processing CreateTablet for tablet 105246590fe24a0eba9b263cf71b18a8 (DEFAULT_TABLE table=test-table [id=814504ca7b48436ebbaee06bb7c3bb04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.041934 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 105246590fe24a0eba9b263cf71b18a8 (DEFAULT_TABLE table=test-table [id=814504ca7b48436ebbaee06bb7c3bb04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.042189 3930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105246590fe24a0eba9b263cf71b18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.042271 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105246590fe24a0eba9b263cf71b18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.042299 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 105246590fe24a0eba9b263cf71b18a8 (DEFAULT_TABLE table=test-table [id=814504ca7b48436ebbaee06bb7c3bb04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.042622 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105246590fe24a0eba9b263cf71b18a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.045193 4284 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Bootstrap starting.
I20260806 22:40:06.045673 4285 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Bootstrap starting.
I20260806 22:40:06.045876 4286 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Bootstrap starting.
I20260806 22:40:06.045953 4284 tablet_bootstrap.cc:654] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.046347 4284 log.cc:824] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:06.046380 4286 tablet_bootstrap.cc:654] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.046583 4285 tablet_bootstrap.cc:654] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.046686 4286 log.cc:824] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:06.046957 4285 log.cc:824] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:06.047173 4284 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: No bootstrap required, opened a new log
I20260806 22:40:06.047241 4284 ts_tablet_manager.cc:1397] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:06.047427 4286 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: No bootstrap required, opened a new log
I20260806 22:40:06.047492 4286 ts_tablet_manager.cc:1397] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:06.047681 4285 tablet_bootstrap.cc:492] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: No bootstrap required, opened a new log
I20260806 22:40:06.047752 4285 ts_tablet_manager.cc:1397] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:06.048940 4286 raft_consensus.cc:348] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.049068 4286 raft_consensus.cc:374] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.049098 4286 raft_consensus.cc:729] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.049242 4286 consensus_queue.cc:260] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.049492 4286 ts_tablet_manager.cc:1428] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:06.049515 3804 heartbeater.cc:503] Master 127.0.186.254:37341 was elected leader, sending a full tablet report...
I20260806 22:40:06.051115 4284 raft_consensus.cc:348] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.051115 4285 raft_consensus.cc:348] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.051292 4284 raft_consensus.cc:374] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.051290 4285 raft_consensus.cc:374] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.051337 4284 raft_consensus.cc:729] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c901f3972249fea2d0d194eb4777c4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.051337 4285 raft_consensus.cc:729] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.051451 4284 consensus_queue.cc:260] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.051451 4285 consensus_queue.cc:260] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.051851 4285 ts_tablet_manager.cc:1428] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:40:06.051877 4035 heartbeater.cc:503] Master 127.0.186.254:37341 was elected leader, sending a full tablet report...
I20260806 22:40:06.052011 4247 heartbeater.cc:503] Master 127.0.186.254:37341 was elected leader, sending a full tablet report...
I20260806 22:40:06.052357 4284 ts_tablet_manager.cc:1428] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
W20260806 22:40:06.094296 4037 tablet.cc:2369] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:06.170614 3805 tablet.cc:2369] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:06.247174 4248 tablet.cc:2369] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:06.309355 4292 raft_consensus.cc:482] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:06.309482 4292 raft_consensus.cc:504] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.309864 4292 leader_election.cc:288] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:06.312801 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.310189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:37017 (local address 127.0.186.195:36017)
0806 22:40:06.310234 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.310235 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.310245 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:06.310318 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.310319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.310328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.310374 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.310496 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.310958 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312030 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.312410 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312473 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.312482 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.312699 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.312744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:06.313043 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.310050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:36017 (local address 127.0.186.193:37017)
0806 22:40:06.310193 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.310203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.310262 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.310384 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.310385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.310387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.310442 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.310444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.311026 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.311027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.311947 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.311950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.312482 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.312604 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.312609 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.312883 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.312931 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260806 22:40:06.313071 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.310181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:48251 (local address 127.0.186.194:39139)
0806 22:40:06.310227 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.310229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.310350 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:40:06.310400 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.310401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.310410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.310470 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.310711 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.311170 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312203 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.312668 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312740 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.312749 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.312971 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.313012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:06.313130 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.310161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:39139 (local address 127.0.186.193:48251)
0806 22:40:06.310272 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.310283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.310346 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.310467 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.310468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.310470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.310633 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.310635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.311247 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.311249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.312109 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.312112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.312749 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.312750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.312861 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.312866 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.312962 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.313002 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260806 22:40:06.313443 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105246590fe24a0eba9b263cf71b18a8" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4" is_pre_election: true
I20260806 22:40:06.313511 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105246590fe24a0eba9b263cf71b18a8" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4" is_pre_election: true
I20260806 22:40:06.313577 4169 raft_consensus.cc:2454] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:06.313678 3960 raft_consensus.cc:2454] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:06.313850 3616 leader_election.cc:302] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, b5da924d8b0344afb1b27013cd2aace4; no voters:
I20260806 22:40:06.313983 4292 raft_consensus.cc:2781] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:06.314039 4292 raft_consensus.cc:482] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:06.314061 4292 raft_consensus.cc:3037] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.314745 4292 raft_consensus.cc:504] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.314872 4292 leader_election.cc:288] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Requested vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:06.315128 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105246590fe24a0eba9b263cf71b18a8" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4"
I20260806 22:40:06.315153 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105246590fe24a0eba9b263cf71b18a8" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4"
I20260806 22:40:06.315220 4169 raft_consensus.cc:3037] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.315213 3960 raft_consensus.cc:3037] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.316156 4169 raft_consensus.cc:2454] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:06.316344 3616 leader_election.cc:302] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, b5da924d8b0344afb1b27013cd2aace4; no voters:
I20260806 22:40:06.316444 4292 raft_consensus.cc:2781] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:06.316601 4292 raft_consensus.cc:686] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Becoming Leader. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Running, Role: LEADER
I20260806 22:40:06.316699 4292 consensus_queue.cc:237] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.316828 3960 raft_consensus.cc:2454] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:06.317485 3474 catalog_manager.cc:5495] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:40:06 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:06.414952 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.408045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.0.186.254:37341)
0806 22:40:06.408129 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.408132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.408270 (+ 138us) server_negotiation.cc:407] Connection header received
0806 22:40:06.408343 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.408344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.408353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.408407 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.408958 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.409450 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.410132 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.410321 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.410455 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.412606 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.412608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.412608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:06.412615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.413867 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.414254 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.414255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.414255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.414304 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.414463 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.414464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.414464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.414564 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.414619 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.414754 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.414854 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":51}
I20260806 22:40:06.415089 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.407986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37341 (local address 127.0.0.1:58854)
0806 22:40:06.408198 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.408208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.408277 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.408427 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.408428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.408806 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:06.408853 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.408856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.409521 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.409522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.410053 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.410055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.410453 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.410454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.410522 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.411038 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:06.411042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:06.412521 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:06.413973 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.413981 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.413983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.414183 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.414356 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.414356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.414357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.414412 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.414616 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:06.414624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:06.414669 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.414728 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.414921 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":89,"thread_start_us":78,"threads_started":1}
I20260806 22:40:06.417102 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:58854:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:40:06.417304 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:06.418730 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 8d89c4c7ac5947fe93dc25b73535da13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66a77eb2abf7476cba1354949ec3822a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:06.418718 3930 tablet_service.cc:1467] Processing CreateTablet for tablet 8d89c4c7ac5947fe93dc25b73535da13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66a77eb2abf7476cba1354949ec3822a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:06.418769 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 8d89c4c7ac5947fe93dc25b73535da13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66a77eb2abf7476cba1354949ec3822a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:06.418898 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d89c4c7ac5947fe93dc25b73535da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.418898 3930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d89c4c7ac5947fe93dc25b73535da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.418901 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d89c4c7ac5947fe93dc25b73535da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.420054 4285 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: Bootstrap starting.
I20260806 22:40:06.420354 4284 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: Bootstrap starting.
I20260806 22:40:06.420651 4285 tablet_bootstrap.cc:654] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.420696 4284 tablet_bootstrap.cc:654] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.420864 4286 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: Bootstrap starting.
I20260806 22:40:06.421277 4286 tablet_bootstrap.cc:654] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.421562 4285 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: No bootstrap required, opened a new log
I20260806 22:40:06.421613 4285 ts_tablet_manager.cc:1397] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:06.421765 4285 raft_consensus.cc:348] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.421833 4285 raft_consensus.cc:374] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.421851 4285 raft_consensus.cc:729] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.421885 4284 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: No bootstrap required, opened a new log
I20260806 22:40:06.421923 4286 tablet_bootstrap.cc:492] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: No bootstrap required, opened a new log
I20260806 22:40:06.421927 4284 ts_tablet_manager.cc:1397] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 22:40:06.421909 4285 consensus_queue.cc:260] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.421958 4286 ts_tablet_manager.cc:1397] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:06.422010 4285 ts_tablet_manager.cc:1428] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:06.422045 4293 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.422065 4284 raft_consensus.cc:348] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.422125 4293 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.422144 4284 raft_consensus.cc:374] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.422164 4284 raft_consensus.cc:729] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c901f3972249fea2d0d194eb4777c4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.422204 4284 consensus_queue.cc:260] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.422228 4286 raft_consensus.cc:348] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.422298 4286 raft_consensus.cc:374] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.422309 4284 ts_tablet_manager.cc:1428] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:06.422326 4286 raft_consensus.cc:729] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.422367 4286 consensus_queue.cc:260] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.422451 4286 ts_tablet_manager.cc:1428] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:06.422545 4294 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.422550 4292 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.422631 4294 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.422631 4292 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.479355 4293 raft_consensus.cc:482] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:06.479442 4293 raft_consensus.cc:504] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.479818 4293 leader_election.cc:288] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865), e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139)
I20260806 22:40:06.482550 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.480122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:43317 (local address 127.0.186.194:39139)
0806 22:40:06.480162 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.480164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.480172 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:06.480201 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.480202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.480212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.480274 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.480444 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.480954 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.481874 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.482231 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.482283 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.482290 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.482449 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.482486 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:06.482551 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.480008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:39139 (local address 127.0.186.195:43317)
0806 22:40:06.480048 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.480061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.480104 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.480302 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.480303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.480305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.480381 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.480383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.481039 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.481040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.481780 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.481782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.482299 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.482301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.482387 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.482392 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.482432 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.482468 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 22:40:06.482595 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.480055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:57437 (local address 127.0.186.193:37865)
0806 22:40:06.480128 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.480130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.480269 (+ 139us) server_negotiation.cc:407] Connection header received
0806 22:40:06.480337 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.480338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.480348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.480405 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:06.480553 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.481034 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.481908 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.482268 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.482338 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.482346 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.482495 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.482538 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:40:06.482640 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.480008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:37865 (local address 127.0.186.195:57437)
0806 22:40:06.480194 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.480204 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.480267 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.480411 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.480412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.480413 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:06.480493 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.480495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.481111 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.481112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.481816 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.481817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.482362 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.482364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.482412 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.482415 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.482465 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.482500 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
I20260806 22:40:06.482817 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d89c4c7ac5947fe93dc25b73535da13" candidate_uuid: "b5da924d8b0344afb1b27013cd2aace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4" is_pre_election: true
I20260806 22:40:06.482914 3960 raft_consensus.cc:2454] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5da924d8b0344afb1b27013cd2aace4 in term 0.
I20260806 22:40:06.482964 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d89c4c7ac5947fe93dc25b73535da13" candidate_uuid: "b5da924d8b0344afb1b27013cd2aace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" is_pre_election: true
I20260806 22:40:06.483078 3732 raft_consensus.cc:2454] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5da924d8b0344afb1b27013cd2aace4 in term 0.
I20260806 22:40:06.483100 4076 leader_election.cc:302] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5da924d8b0344afb1b27013cd2aace4, e7c901f3972249fea2d0d194eb4777c4; no voters:
I20260806 22:40:06.483244 4293 raft_consensus.cc:2781] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:06.483299 4293 raft_consensus.cc:482] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:06.483327 4293 raft_consensus.cc:3037] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.484133 4293 raft_consensus.cc:504] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.484256 4293 leader_election.cc:288] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [CANDIDATE]: Term 1 election: Requested vote from peers 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865), e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139)
I20260806 22:40:06.484395 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d89c4c7ac5947fe93dc25b73535da13" candidate_uuid: "b5da924d8b0344afb1b27013cd2aace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04"
I20260806 22:40:06.484463 3732 raft_consensus.cc:3037] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.484444 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d89c4c7ac5947fe93dc25b73535da13" candidate_uuid: "b5da924d8b0344afb1b27013cd2aace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4"
I20260806 22:40:06.484522 3960 raft_consensus.cc:3037] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.485095 3732 raft_consensus.cc:2454] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5da924d8b0344afb1b27013cd2aace4 in term 1.
I20260806 22:40:06.485124 3960 raft_consensus.cc:2454] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5da924d8b0344afb1b27013cd2aace4 in term 1.
I20260806 22:40:06.485270 4072 leader_election.cc:302] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, b5da924d8b0344afb1b27013cd2aace4; no voters:
I20260806 22:40:06.485366 4293 raft_consensus.cc:2781] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:06.485519 4293 raft_consensus.cc:686] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [term 1 LEADER]: Becoming Leader. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Running, Role: LEADER
I20260806 22:40:06.485643 4293 consensus_queue.cc:237] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.486085 4316 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: TxnStatusTablet state changed. Reason: New leader b5da924d8b0344afb1b27013cd2aace4. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.486171 4316 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:06.486341 4320 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:40:06.486366 3473 catalog_manager.cc:5495] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:06.486462 4320 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:40:06.496474 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.494188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.0.186.195:36017)
0806 22:40:06.494231 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.494233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.494306 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:40:06.494380 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.494381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.494397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.494452 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:06.494560 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.495017 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.495758 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.495947 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.496000 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.496171 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.496259 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.496324 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.496375 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.496394 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:06.496476 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.494139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:36017 (local address 127.0.0.1:38174)
0806 22:40:06.494253 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.494264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.494318 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.494464 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.494465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.494466 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:06.494510 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.494512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.495094 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.495095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.495687 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.495689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.496007 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.496008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.496074 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.496097 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.496314 (+ 217us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.496322 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.496362 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.496412 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 22:40:06.498235 3732 raft_consensus.cc:1261] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Refusing update from remote peer b5da924d8b0344afb1b27013cd2aace4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.498262 3960 raft_consensus.cc:1261] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Refusing update from remote peer b5da924d8b0344afb1b27013cd2aace4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.499441 4316 consensus_queue.cc:1035] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:06.499569 4293 consensus_queue.cc:1035] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:06.500718 4294 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: TxnStatusTablet state changed. Reason: New leader b5da924d8b0344afb1b27013cd2aace4. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.500731 4292 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: TxnStatusTablet state changed. Reason: New leader b5da924d8b0344afb1b27013cd2aace4. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.500801 4294 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.500811 4292 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.504045 4293 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.504129 4293 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:06.504216 4293 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.504267 4293 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P b5da924d8b0344afb1b27013cd2aace4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:06.504644 4294 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.504726 4294 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P e7c901f3972249fea2d0d194eb4777c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.505084 4292 tablet_replica.cc:439] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5da924d8b0344afb1b27013cd2aace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } } }
I20260806 22:40:06.505156 4292 tablet_replica.cc:442] T 8d89c4c7ac5947fe93dc25b73535da13 P 9c42946b6e3c4cacadb3a4ad8489ff04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:06.505442 4330 mvcc.cc:205] Tried to move back new op lower bound from 7315685402614755328 to 7315685402565799936. Current Snapshot: MvccSnapshot[applied={T|T < 7315685402614755328}]
I20260806 22:40:06.508744 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.506332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.0.186.193:37865)
0806 22:40:06.506366 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.506368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.506376 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:06.506450 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.506451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.506464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.506507 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:06.506616 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.507055 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.508151 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.508325 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.508367 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.508503 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.508585 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.508632 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.508681 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.508699 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:06.513649 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.511136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54695 (local address 127.0.186.195:36017)
0806 22:40:06.511184 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.511186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.511243 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:40:06.511333 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.511334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.511344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.511385 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:06.511495 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.512000 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.513092 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.513260 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.513305 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.513494 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.513515 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.513544 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.513591 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.513605 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:06.513720 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.511034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:36017 (local address 127.0.186.193:54695)
0806 22:40:06.511186 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.511196 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.511258 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.511389 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.511390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.511391 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:06.511445 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.511446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.512061 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.512063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.513014 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.513017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.513324 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.513325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.513396 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.513412 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.513548 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.513550 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.513590 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.513628 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260806 22:40:06.515697 3960 raft_consensus.cc:1261] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.515786 4169 raft_consensus.cc:1261] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.516045 4299 consensus_queue.cc:1035] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:06.516194 4299 consensus_queue.cc:1035] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:06.517792 4329 mvcc.cc:205] Tried to move back new op lower bound from 7315685402686234624 to 7315685401873788928. Current Snapshot: MvccSnapshot[applied={T|T < 7315685402686234624}]
I20260806 22:40:06.517831 4328 mvcc.cc:205] Tried to move back new op lower bound from 7315685402686234624 to 7315685401873788928. Current Snapshot: MvccSnapshot[applied={T|T < 7315685402686234624}]
I20260806 22:40:06.526225 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.521782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:58307 (local address 127.0.186.193:37865)
0806 22:40:06.521818 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.521819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.523444 (+ 1625us) server_negotiation.cc:407] Connection header received
0806 22:40:06.523514 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.523515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.523525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.523568 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:06.523740 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.524434 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.525530 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.525775 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.525829 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.525991 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.526023 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.526067 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.526139 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.526162 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:06.526242 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.521764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:37865 (local address 127.0.186.195:58307)
0806 22:40:06.523374 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.523384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.523453 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.523611 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.523612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.523614 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:06.523672 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.523673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.524528 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.524529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.525455 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.525457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.525841 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.525842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.525914 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.525930 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:06.526067 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:06.526068 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.526119 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.526159 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1570,"thread_start_us":79,"threads_started":1}
I20260806 22:40:06.533656 3474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:06.533747 3474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:06.534384 3474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Sending DeleteTablet for 3 replicas of tablet 105246590fe24a0eba9b263cf71b18a8
I20260806 22:40:06.534686 3930 tablet_service.cc:1514] Processing DeleteTablet for tablet 105246590fe24a0eba9b263cf71b18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:06 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:50916
I20260806 22:40:06.534775 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 105246590fe24a0eba9b263cf71b18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:06 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40974
I20260806 22:40:06.535032 4356 tablet_replica.cc:330] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: stopping tablet replica
I20260806 22:40:06.535185 4356 raft_consensus.cc:2229] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:06.535225 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 105246590fe24a0eba9b263cf71b18a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:06 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:38172
I20260806 22:40:06.535353 4356 raft_consensus.cc:2258] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:06.535806 4356 ts_tablet_manager.cc:1905] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:06.535792 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:06.535938 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:06.537264 3930 tablet_service.cc:1467] Processing CreateTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 (DEFAULT_TABLE table=test-table [id=67635c8778124b289585ada7a7c19847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.537371 3930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a74855b3ad45829d957a2c22d5bca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.537956 4356 ts_tablet_manager.cc:1918] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:06.538020 4356 log.cc:1196] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/105246590fe24a0eba9b263cf71b18a8
I20260806 22:40:06.538276 4356 ts_tablet_manager.cc:1939] T 105246590fe24a0eba9b263cf71b18a8 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting consensus metadata
I20260806 22:40:06.539014 4284 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Bootstrap starting.
I20260806 22:40:06.539471 3452 catalog_manager.cc:4841] TS 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865): tablet 105246590fe24a0eba9b263cf71b18a8 (table test-table [id=814504ca7b48436ebbaee06bb7c3bb04]) successfully deleted
I20260806 22:40:06.539778 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 (DEFAULT_TABLE table=test-table [id=67635c8778124b289585ada7a7c19847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.539875 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a74855b3ad45829d957a2c22d5bca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.539945 4284 tablet_bootstrap.cc:654] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.541270 4284 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: No bootstrap required, opened a new log
I20260806 22:40:06.541312 4284 ts_tablet_manager.cc:1397] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:40:06.541407 4355 tablet_replica.cc:330] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: stopping tablet replica
I20260806 22:40:06.541529 4355 raft_consensus.cc:2229] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:06.541586 4355 raft_consensus.cc:2258] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:06.541661 4286 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Bootstrap starting.
I20260806 22:40:06.541680 4284 raft_consensus.cc:348] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.541785 4284 raft_consensus.cc:374] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.541810 4284 raft_consensus.cc:729] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c901f3972249fea2d0d194eb4777c4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.541887 4284 consensus_queue.cc:260] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.541978 4284 ts_tablet_manager.cc:1428] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:06.542644 4286 tablet_bootstrap.cc:654] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.543792 4286 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: No bootstrap required, opened a new log
I20260806 22:40:06.543836 4286 ts_tablet_manager.cc:1397] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:40:06.544198 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 (DEFAULT_TABLE table=test-table [id=67635c8778124b289585ada7a7c19847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:06.544308 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a74855b3ad45829d957a2c22d5bca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.544332 4286 raft_consensus.cc:348] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.544394 4286 raft_consensus.cc:374] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.544410 4286 raft_consensus.cc:729] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.544450 4286 consensus_queue.cc:260] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.544530 4286 ts_tablet_manager.cc:1428] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:06.545758 4285 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Bootstrap starting.
I20260806 22:40:06.546233 4285 tablet_bootstrap.cc:654] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.547407 4285 tablet_bootstrap.cc:492] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: No bootstrap required, opened a new log
I20260806 22:40:06.547478 4285 ts_tablet_manager.cc:1397] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:06.547762 4285 raft_consensus.cc:348] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.547843 4285 raft_consensus.cc:374] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.547863 4285 raft_consensus.cc:729] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.547899 4285 consensus_queue.cc:260] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.547991 4285 ts_tablet_manager.cc:1428] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:06.551429 4357 tablet_replica.cc:330] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: stopping tablet replica
I20260806 22:40:06.551611 4357 raft_consensus.cc:2229] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:06.551646 4355 ts_tablet_manager.cc:1905] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:06.551760 4357 raft_consensus.cc:2258] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:06.552130 4357 ts_tablet_manager.cc:1905] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:06.553717 4355 ts_tablet_manager.cc:1918] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:06.553789 4355 log.cc:1196] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/105246590fe24a0eba9b263cf71b18a8
I20260806 22:40:06.554047 4355 ts_tablet_manager.cc:1939] T 105246590fe24a0eba9b263cf71b18a8 P e7c901f3972249fea2d0d194eb4777c4: Deleting consensus metadata
I20260806 22:40:06.554520 3454 catalog_manager.cc:4841] TS e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139): tablet 105246590fe24a0eba9b263cf71b18a8 (table test-table [id=814504ca7b48436ebbaee06bb7c3bb04]) successfully deleted
I20260806 22:40:06.554679 4357 ts_tablet_manager.cc:1918] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:06.554728 4357 log.cc:1196] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/105246590fe24a0eba9b263cf71b18a8
I20260806 22:40:06.554875 4357 ts_tablet_manager.cc:1939] T 105246590fe24a0eba9b263cf71b18a8 P b5da924d8b0344afb1b27013cd2aace4: Deleting consensus metadata
I20260806 22:40:06.555370 3456 catalog_manager.cc:4841] TS b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017): tablet 105246590fe24a0eba9b263cf71b18a8 (table test-table [id=814504ca7b48436ebbaee06bb7c3bb04]) successfully deleted
I20260806 22:40:06.663218 4299 raft_consensus.cc:482] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:06.663336 4299 raft_consensus.cc:504] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.663496 4299 leader_election.cc:288] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:06.663786 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a74855b3ad45829d957a2c22d5bca3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4" is_pre_election: true
I20260806 22:40:06.663880 4169 raft_consensus.cc:2454] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:06.664094 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a74855b3ad45829d957a2c22d5bca3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4" is_pre_election: true
I20260806 22:40:06.664177 3960 raft_consensus.cc:2454] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:06.664764 3615 leader_election.cc:302] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, e7c901f3972249fea2d0d194eb4777c4; no voters:
I20260806 22:40:06.664971 4299 raft_consensus.cc:2781] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:06.665010 4299 raft_consensus.cc:482] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:06.665030 4299 raft_consensus.cc:3037] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.667104 4299 raft_consensus.cc:504] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.667244 4299 leader_election.cc:288] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Requested vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:06.667477 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a74855b3ad45829d957a2c22d5bca3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4"
I20260806 22:40:06.667554 4169 raft_consensus.cc:3037] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.668321 4169 raft_consensus.cc:2454] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:06.668514 3616 leader_election.cc:302] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, b5da924d8b0344afb1b27013cd2aace4; no voters:
I20260806 22:40:06.668789 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a74855b3ad45829d957a2c22d5bca3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4"
I20260806 22:40:06.668865 3960 raft_consensus.cc:3037] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.669606 3960 raft_consensus.cc:2454] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:06.670271 4299 raft_consensus.cc:2781] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:06.670333 4299 raft_consensus.cc:686] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Becoming Leader. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Running, Role: LEADER
I20260806 22:40:06.670377 4299 consensus_queue.cc:237] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:06.671140 3474 catalog_manager.cc:5495] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:06.706782 3958 raft_consensus.cc:1261] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.707063 4299 consensus_queue.cc:1035] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:06.707185 4169 raft_consensus.cc:1261] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:06.707428 4299 consensus_queue.cc:1035] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:40:06.712451 4330 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:40:08.231133 3474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:08.231236 3474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:08.232038 3474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Sending DeleteTablet for 3 replicas of tablet 46a74855b3ad45829d957a2c22d5bca3
I20260806 22:40:08.232270 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40974
I20260806 22:40:08.232333 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:38172
I20260806 22:40:08.232313 3930 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a74855b3ad45829d957a2c22d5bca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:50916
I20260806 22:40:08.232460 4397 tablet_replica.cc:330] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: stopping tablet replica
I20260806 22:40:08.232511 4397 raft_consensus.cc:2229] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:08.232510 4398 tablet_replica.cc:330] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: stopping tablet replica
I20260806 22:40:08.232558 4398 raft_consensus.cc:2229] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:08.232621 4398 raft_consensus.cc:2258] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.232622 4397 raft_consensus.cc:2258] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.232800 4399 tablet_replica.cc:330] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: stopping tablet replica
I20260806 22:40:08.232861 4399 raft_consensus.cc:2229] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:08.232904 4398 ts_tablet_manager.cc:1905] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.232928 4399 raft_consensus.cc:2258] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.233145 4399 ts_tablet_manager.cc:1905] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.235492 4397 ts_tablet_manager.cc:1905] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.243427 4397 ts_tablet_manager.cc:1918] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:40:08.243435 4399 ts_tablet_manager.cc:1918] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:40:08.243499 4399 log.cc:1196] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/46a74855b3ad45829d957a2c22d5bca3
I20260806 22:40:08.243500 4397 log.cc:1196] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/46a74855b3ad45829d957a2c22d5bca3
I20260806 22:40:08.243714 4399 ts_tablet_manager.cc:1939] T 46a74855b3ad45829d957a2c22d5bca3 P e7c901f3972249fea2d0d194eb4777c4: Deleting consensus metadata
I20260806 22:40:08.243718 4397 ts_tablet_manager.cc:1939] T 46a74855b3ad45829d957a2c22d5bca3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting consensus metadata
Aug 06 22:40:08 dist-test-slave-gjrz krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260806 22:40:08.247519 4398 ts_tablet_manager.cc:1918] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:40:08.247581 4398 log.cc:1196] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/46a74855b3ad45829d957a2c22d5bca3
I20260806 22:40:08.247776 4398 ts_tablet_manager.cc:1939] T 46a74855b3ad45829d957a2c22d5bca3 P b5da924d8b0344afb1b27013cd2aace4: Deleting consensus metadata
I20260806 22:40:08.249928 3452 catalog_manager.cc:4841] TS 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865): tablet 46a74855b3ad45829d957a2c22d5bca3 (table test-table [id=67635c8778124b289585ada7a7c19847]) successfully deleted
I20260806 22:40:08.249940 3454 catalog_manager.cc:4841] TS e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139): tablet 46a74855b3ad45829d957a2c22d5bca3 (table test-table [id=67635c8778124b289585ada7a7c19847]) successfully deleted
I20260806 22:40:08.253273 3456 catalog_manager.cc:4841] TS b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017): tablet 46a74855b3ad45829d957a2c22d5bca3 (table test-table [id=67635c8778124b289585ada7a7c19847]) successfully deleted
Aug 06 22:40:08 dist-test-slave-gjrz krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:08.269945 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:08.259846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.0.186.254:37341)
0806 22:40:08.259981 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:08.259983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:08.259994 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:08.260054 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:08.260055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:08.260066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:08.260127 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:08.260675 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:08.261184 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:08.261911 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:08.262116 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:08.262169 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:08.267482 (+ 5313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:08.267484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:08.267485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:08.267492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:08.268831 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:08.269176 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:08.269177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:08.269178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:08.269243 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:08.269408 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:08.269408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:08.269409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:08.269523 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:08.269647 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:08.269729 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:40:08.269819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260806 22:40:08.271258 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:08.271457 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:08.272835 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 4083deed7064425086629defdd3681e3 (DEFAULT_TABLE table=test-table [id=bda0d8afe262483583a63ad61a1f460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.272861 3930 tablet_service.cc:1467] Processing CreateTablet for tablet 4083deed7064425086629defdd3681e3 (DEFAULT_TABLE table=test-table [id=bda0d8afe262483583a63ad61a1f460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.272966 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4083deed7064425086629defdd3681e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.273007 3930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4083deed7064425086629defdd3681e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.274070 4422 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Bootstrap starting.
I20260806 22:40:08.274127 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 4083deed7064425086629defdd3681e3 (DEFAULT_TABLE table=test-table [id=bda0d8afe262483583a63ad61a1f460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.274236 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4083deed7064425086629defdd3681e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.274767 4422 tablet_bootstrap.cc:654] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.275744 4423 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Bootstrap starting.
I20260806 22:40:08.275835 4422 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: No bootstrap required, opened a new log
I20260806 22:40:08.275883 4422 ts_tablet_manager.cc:1397] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:08.276149 4423 tablet_bootstrap.cc:654] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.276374 4422 raft_consensus.cc:348] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.276463 4422 raft_consensus.cc:374] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.276489 4422 raft_consensus.cc:729] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c901f3972249fea2d0d194eb4777c4, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.276571 4422 consensus_queue.cc:260] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.276753 4423 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: No bootstrap required, opened a new log
I20260806 22:40:08.276819 4422 ts_tablet_manager.cc:1428] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:08.276849 4423 ts_tablet_manager.cc:1397] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:08.277037 4423 raft_consensus.cc:348] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.277114 4423 raft_consensus.cc:374] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.277134 4423 raft_consensus.cc:729] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.277201 4423 consensus_queue.cc:260] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.277324 4423 ts_tablet_manager.cc:1428] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:08.277779 4425 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Bootstrap starting.
I20260806 22:40:08.278155 4425 tablet_bootstrap.cc:654] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.278667 4425 tablet_bootstrap.cc:492] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: No bootstrap required, opened a new log
I20260806 22:40:08.278710 4425 ts_tablet_manager.cc:1397] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:08.278836 4425 raft_consensus.cc:348] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.278891 4425 raft_consensus.cc:374] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.278906 4425 raft_consensus.cc:729] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.278949 4425 consensus_queue.cc:260] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.279031 4425 ts_tablet_manager.cc:1428] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:08.615072 4392 raft_consensus.cc:482] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:08.615188 4392 raft_consensus.cc:504] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.615388 4392 leader_election.cc:288] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:08.615625 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4083deed7064425086629defdd3681e3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4" is_pre_election: true
I20260806 22:40:08.615626 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4083deed7064425086629defdd3681e3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4" is_pre_election: true
I20260806 22:40:08.615743 4169 raft_consensus.cc:2454] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:08.615743 3958 raft_consensus.cc:2454] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:08.615926 3616 leader_election.cc:302] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, b5da924d8b0344afb1b27013cd2aace4; no voters:
I20260806 22:40:08.616029 4392 raft_consensus.cc:2781] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:08.616072 4392 raft_consensus.cc:482] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:08.616086 4392 raft_consensus.cc:3037] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.653009 4392 raft_consensus.cc:504] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.653221 4392 leader_election.cc:288] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Requested vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:08.653434 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4083deed7064425086629defdd3681e3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4"
I20260806 22:40:08.653529 3958 raft_consensus.cc:3037] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.653535 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4083deed7064425086629defdd3681e3" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4"
I20260806 22:40:08.653644 4169 raft_consensus.cc:3037] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.654197 3958 raft_consensus.cc:2454] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:08.654367 3615 leader_election.cc:302] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, e7c901f3972249fea2d0d194eb4777c4; no voters:
I20260806 22:40:08.654460 4392 raft_consensus.cc:2781] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:08.654510 4392 raft_consensus.cc:686] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Becoming Leader. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Running, Role: LEADER
I20260806 22:40:08.654533 4169 raft_consensus.cc:2454] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:08.654551 4392 consensus_queue.cc:237] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.655179 3474 catalog_manager.cc:5495] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:08.719878 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:08.717422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.0.186.193:37865)
0806 22:40:08.717628 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:08.717630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:08.717638 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:08.717663 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:08.717664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:08.717679 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:08.717737 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:08.717901 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:08.718469 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:08.719235 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:08.719513 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:08.719569 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:08.719696 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:08.719723 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:08.719759 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:08.719796 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:40:08.719817 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":137,"threads_started":1}
I20260806 22:40:08.721963 4169 raft_consensus.cc:1261] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:08.721989 3958 raft_consensus.cc:1261] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:08.722174 4392 consensus_queue.cc:1035] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:08.722293 4382 consensus_queue.cc:1035] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:08.731699 3474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:08.731791 3474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:08.732295 3474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Sending DeleteTablet for 3 replicas of tablet 4083deed7064425086629defdd3681e3
I20260806 22:40:08.732561 3930 tablet_service.cc:1514] Processing DeleteTablet for tablet 4083deed7064425086629defdd3681e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:50916
I20260806 22:40:08.732561 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 4083deed7064425086629defdd3681e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:38172
I20260806 22:40:08.732659 4399 tablet_replica.cc:330] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: stopping tablet replica
I20260806 22:40:08.732707 4399 raft_consensus.cc:2229] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:08.732743 4399 raft_consensus.cc:2258] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.732803 4398 tablet_replica.cc:330] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: stopping tablet replica
I20260806 22:40:08.732859 4398 raft_consensus.cc:2229] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:08.732897 4398 raft_consensus.cc:2258] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.732963 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 4083deed7064425086629defdd3681e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:08 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40974
I20260806 22:40:08.733021 4399 ts_tablet_manager.cc:1905] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.733040 4397 tablet_replica.cc:330] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: stopping tablet replica
I20260806 22:40:08.733095 4397 raft_consensus.cc:2229] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:08.733141 4397 raft_consensus.cc:2258] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:08.733282 4398 ts_tablet_manager.cc:1905] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.733418 4397 ts_tablet_manager.cc:1905] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:08.734239 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:08.734369 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:08.735486 3705 tablet_service.cc:1467] Processing CreateTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 (DEFAULT_TABLE table=test-table [id=538331b34bc24dfc92d5fe174a7d1064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.735527 3930 tablet_service.cc:1467] Processing CreateTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 (DEFAULT_TABLE table=test-table [id=538331b34bc24dfc92d5fe174a7d1064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.735705 4143 tablet_service.cc:1467] Processing CreateTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 (DEFAULT_TABLE table=test-table [id=538331b34bc24dfc92d5fe174a7d1064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:08.752049 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f44ec3206ccc4fbc97b3fefb6a99c838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.752049 3930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f44ec3206ccc4fbc97b3fefb6a99c838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.753350 4422 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Bootstrap starting.
I20260806 22:40:08.753350 4425 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Bootstrap starting.
I20260806 22:40:08.753789 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f44ec3206ccc4fbc97b3fefb6a99c838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:08.753989 4422 tablet_bootstrap.cc:654] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.753989 4425 tablet_bootstrap.cc:654] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.754662 4425 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: No bootstrap required, opened a new log
I20260806 22:40:08.754709 4425 ts_tablet_manager.cc:1397] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:08.754820 4425 raft_consensus.cc:348] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.754885 4425 raft_consensus.cc:374] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.754904 4425 raft_consensus.cc:729] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da924d8b0344afb1b27013cd2aace4, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.754949 4425 consensus_queue.cc:260] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.755038 4425 ts_tablet_manager.cc:1428] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:08.755192 4422 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: No bootstrap required, opened a new log
I20260806 22:40:08.755239 4422 ts_tablet_manager.cc:1397] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:08.755662 4422 raft_consensus.cc:348] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.755764 4422 raft_consensus.cc:374] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.755784 4422 raft_consensus.cc:729] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c901f3972249fea2d0d194eb4777c4, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.755790 4423 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04: Bootstrap starting.
I20260806 22:40:08.755833 4422 consensus_queue.cc:260] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.755918 4422 ts_tablet_manager.cc:1428] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:08.756173 4423 tablet_bootstrap.cc:654] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:08.756753 4423 tablet_bootstrap.cc:492] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04: No bootstrap required, opened a new log
I20260806 22:40:08.756795 4423 ts_tablet_manager.cc:1397] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:08.756939 4423 raft_consensus.cc:348] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.757019 4423 raft_consensus.cc:374] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:08.757040 4423 raft_consensus.cc:729] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Initialized, Role: FOLLOWER
I20260806 22:40:08.757076 4423 consensus_queue.cc:260] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.757166 4423 ts_tablet_manager.cc:1428] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:08.757547 4399 ts_tablet_manager.cc:1918] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:08.757550 4398 ts_tablet_manager.cc:1918] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:08.757606 4399 log.cc:1196] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/4083deed7064425086629defdd3681e3
I20260806 22:40:08.757609 4398 log.cc:1196] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/4083deed7064425086629defdd3681e3
I20260806 22:40:08.757817 4398 ts_tablet_manager.cc:1939] T 4083deed7064425086629defdd3681e3 P b5da924d8b0344afb1b27013cd2aace4: Deleting consensus metadata
I20260806 22:40:08.757844 4399 ts_tablet_manager.cc:1939] T 4083deed7064425086629defdd3681e3 P e7c901f3972249fea2d0d194eb4777c4: Deleting consensus metadata
I20260806 22:40:08.788551 3454 catalog_manager.cc:4841] TS e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139): tablet 4083deed7064425086629defdd3681e3 (table test-table [id=bda0d8afe262483583a63ad61a1f460b]) successfully deleted
I20260806 22:40:08.831771 3456 catalog_manager.cc:4841] TS b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017): tablet 4083deed7064425086629defdd3681e3 (table test-table [id=bda0d8afe262483583a63ad61a1f460b]) successfully deleted
I20260806 22:40:08.846287 4397 ts_tablet_manager.cc:1918] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:08.846381 4397 log.cc:1196] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/4083deed7064425086629defdd3681e3
I20260806 22:40:08.846611 4397 ts_tablet_manager.cc:1939] T 4083deed7064425086629defdd3681e3 P 9c42946b6e3c4cacadb3a4ad8489ff04: Deleting consensus metadata
I20260806 22:40:08.848675 3452 catalog_manager.cc:4841] TS 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865): tablet 4083deed7064425086629defdd3681e3 (table test-table [id=bda0d8afe262483583a63ad61a1f460b]) successfully deleted
I20260806 22:40:08.880542 4292 raft_consensus.cc:482] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:08.880638 4292 raft_consensus.cc:504] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.880813 4292 leader_election.cc:288] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:08.881062 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f44ec3206ccc4fbc97b3fefb6a99c838" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4" is_pre_election: true
I20260806 22:40:08.881062 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f44ec3206ccc4fbc97b3fefb6a99c838" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4" is_pre_election: true
I20260806 22:40:08.881175 4169 raft_consensus.cc:2454] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:08.881175 3958 raft_consensus.cc:2454] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 0.
I20260806 22:40:08.881332 3615 leader_election.cc:302] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, e7c901f3972249fea2d0d194eb4777c4; no voters:
I20260806 22:40:08.881438 4292 raft_consensus.cc:2781] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:08.881501 4292 raft_consensus.cc:482] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:08.881520 4292 raft_consensus.cc:3037] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.946341 4292 raft_consensus.cc:504] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.946578 4292 leader_election.cc:288] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Requested vote from peers e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139), b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017)
I20260806 22:40:08.946882 4169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f44ec3206ccc4fbc97b3fefb6a99c838" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5da924d8b0344afb1b27013cd2aace4"
I20260806 22:40:08.946885 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f44ec3206ccc4fbc97b3fefb6a99c838" candidate_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c901f3972249fea2d0d194eb4777c4"
I20260806 22:40:08.946988 4169 raft_consensus.cc:3037] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.947021 3958 raft_consensus.cc:3037] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:08.947837 3958 raft_consensus.cc:2454] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:08.947837 4169 raft_consensus.cc:2454] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c42946b6e3c4cacadb3a4ad8489ff04 in term 1.
I20260806 22:40:08.948053 3615 leader_election.cc:302] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c42946b6e3c4cacadb3a4ad8489ff04, e7c901f3972249fea2d0d194eb4777c4; no voters:
I20260806 22:40:08.948176 4292 raft_consensus.cc:2781] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:08.948221 4292 raft_consensus.cc:686] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [term 1 LEADER]: Becoming Leader. State: Replica: 9c42946b6e3c4cacadb3a4ad8489ff04, State: Running, Role: LEADER
I20260806 22:40:08.948267 4292 consensus_queue.cc:237] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } }
I20260806 22:40:08.948913 3474 catalog_manager.cc:5495] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c42946b6e3c4cacadb3a4ad8489ff04" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 37865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:08.970144 4169 raft_consensus.cc:1261] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:08.970144 3958 raft_consensus.cc:1261] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Refusing update from remote peer 9c42946b6e3c4cacadb3a4ad8489ff04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:08.970369 4392 consensus_queue.cc:1035] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5da924d8b0344afb1b27013cd2aace4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:08.970490 4292 consensus_queue.cc:1035] T f44ec3206ccc4fbc97b3fefb6a99c838 P 9c42946b6e3c4cacadb3a4ad8489ff04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c901f3972249fea2d0d194eb4777c4" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:40:08.974725 4374 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:40:10.490842 3474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:10.490948 3474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:10.491715 3474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18d3a384b5494cbaa15e38d751574428: Sending DeleteTablet for 3 replicas of tablet f44ec3206ccc4fbc97b3fefb6a99c838
I20260806 22:40:10.491961 3930 tablet_service.cc:1514] Processing DeleteTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:10 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:50916
I20260806 22:40:10.492003 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:10 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:38172
I20260806 22:40:10.492086 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3588
I20260806 22:40:10.492152 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet f44ec3206ccc4fbc97b3fefb6a99c838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:10 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40974
I20260806 22:40:10.501359 4479 tablet_replica.cc:330] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: stopping tablet replica
I20260806 22:40:10.501477 4479 raft_consensus.cc:2229] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:10.501528 4479 raft_consensus.cc:2258] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:10.501812 4479 ts_tablet_manager.cc:1905] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:10.503283 4478 tablet_replica.cc:330] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: stopping tablet replica
I20260806 22:40:10.503391 4479 ts_tablet_manager.cc:1918] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:40:10.503427 4478 raft_consensus.cc:2229] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:10.503454 4479 log.cc:1196] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/f44ec3206ccc4fbc97b3fefb6a99c838
I20260806 22:40:10.503471 4478 raft_consensus.cc:2258] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:10.503669 4479 ts_tablet_manager.cc:1939] T f44ec3206ccc4fbc97b3fefb6a99c838 P b5da924d8b0344afb1b27013cd2aace4: Deleting consensus metadata
I20260806 22:40:10.503741 4478 ts_tablet_manager.cc:1905] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:10.504019 3456 catalog_manager.cc:4841] TS b5da924d8b0344afb1b27013cd2aace4 (127.0.186.195:36017): tablet f44ec3206ccc4fbc97b3fefb6a99c838 (table test-table [id=538331b34bc24dfc92d5fe174a7d1064]) successfully deleted
I20260806 22:40:10.504865 4478 ts_tablet_manager.cc:1918] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:40:10.504917 4478 log.cc:1196] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/f44ec3206ccc4fbc97b3fefb6a99c838
I20260806 22:40:10.505084 4478 ts_tablet_manager.cc:1939] T f44ec3206ccc4fbc97b3fefb6a99c838 P e7c901f3972249fea2d0d194eb4777c4: Deleting consensus metadata
I20260806 22:40:10.505429 3454 catalog_manager.cc:4841] TS e7c901f3972249fea2d0d194eb4777c4 (127.0.186.194:39139): tablet f44ec3206ccc4fbc97b3fefb6a99c838 (table test-table [id=538331b34bc24dfc92d5fe174a7d1064]) successfully deleted
W20260806 22:40:10.506843 3452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:37865 (error 108)
I20260806 22:40:10.506886 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3818
I20260806 22:40:10.514633 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:10.511180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:37865 (local address 127.0.0.1:46540)
0806 22:40:10.514546 (+ 3366us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:10.514597 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:37865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3340,"thread_start_us":553,"threads_started":1}
W20260806 22:40:10.514737 3452 catalog_manager.cc:4584] TS 9c42946b6e3c4cacadb3a4ad8489ff04 (127.0.186.193:37865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f44ec3206ccc4fbc97b3fefb6a99c838: Network error: Client connection negotiation failed: client connection to 127.0.186.193:37865: connect: Connection refused (error 111)
I20260806 22:40:10.520085 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 4053
I20260806 22:40:10.529533 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3417
2026-08-06T22:40:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6456 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:10 dist-test-slave-gjrz krb5kdc[4486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:10 dist-test-slave-gjrz krb5kdc[4486](info): set up 2 sockets
Aug 06 22:40:10 dist-test-slave-gjrz krb5kdc[4486](info): commencing operation
krb5kdc: starting...
W20260806 22:40:11.672091 747 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:11 dist-test-slave-gjrz krb5kdc[4486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:11.783730 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:44199
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:44199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:11.862823 4508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:11.863018 4508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:11.863039 4508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:11.864200 4508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:11.864240 4508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:11.864253 4508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:11.864264 4508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:11.864275 4508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:11.865782 4508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:44199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:44199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:11.866009 4508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:11.866218 4508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:11.871037 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:11.871043 4520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:11.871094 4508 server_base.cc:1034] running on GCE node
W20260806 22:40:11.871049 4516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:11.871430 4508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:11.871727 4508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:11.872869 4508 hybrid_clock.cc:648] HybridClock initialized: now 1786056011872851 us; error 32 us; skew 500 ppm
Aug 06 22:40:11 dist-test-slave-gjrz krb5kdc[4486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:11.880545 4508 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:11.880923 4508 webserver.cc:460] Webserver started at http://127.0.186.254:35203/ using document root <none> and password file <none>
I20260806 22:40:11.881132 4508 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:11.881171 4508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:11.881253 4508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:11.882232 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "7b916250fc3342458c430e8d1357d03f"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-gjrz"
server_key: "3013a508f8ff6aaff06b7adec0e56c06"
server_key_iv: "7057104e122c98ca334c86341a0eb18c"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.882519 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "7b916250fc3342458c430e8d1357d03f"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-gjrz"
server_key: "3013a508f8ff6aaff06b7adec0e56c06"
server_key_iv: "7057104e122c98ca334c86341a0eb18c"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.883783 4508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:11.884459 4532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:11.884616 4508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:11.884701 4508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "7b916250fc3342458c430e8d1357d03f"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-gjrz"
server_key: "3013a508f8ff6aaff06b7adec0e56c06"
server_key_iv: "7057104e122c98ca334c86341a0eb18c"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.884756 4508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:11.896016 4508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:11.897645 4508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:11.897763 4508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:11.901577 4508 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:44199
I20260806 22:40:11.901636 4615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:44199 every 8 connection(s)
I20260806 22:40:11.902150 4508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:11.902825 4619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:11.905016 4619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f: Bootstrap starting.
I20260806 22:40:11.905594 4619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:11.905851 4619 log.cc:824] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:11.906600 4619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f: No bootstrap required, opened a new log
I20260806 22:40:11.907994 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 4508
I20260806 22:40:11.908090 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:11.908268 747 external_mini_cluster.cc:1428] Setting key 1a398f22d2d54085da4150f4eacf462c
I20260806 22:40:11.908469 4619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } }
I20260806 22:40:11.908618 4619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:11.908689 4619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b916250fc3342458c430e8d1357d03f, State: Initialized, Role: FOLLOWER
I20260806 22:40:11.908850 4619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } }
I20260806 22:40:11.908943 4619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:11.908974 4619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:11.909024 4619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:11.909636 4619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } }
I20260806 22:40:11.909736 4619 leader_election.cc:302] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b916250fc3342458c430e8d1357d03f; no voters:
I20260806 22:40:11.909889 4619 leader_election.cc:288] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:11.909929 4626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:11.910099 4626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [term 1 LEADER]: Becoming Leader. State: Replica: 7b916250fc3342458c430e8d1357d03f, State: Running, Role: LEADER
I20260806 22:40:11.910156 4619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:11.910210 4626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } }
I20260806 22:40:11.910537 4626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b916250fc3342458c430e8d1357d03f. Latest consensus state: current_term: 1 leader_uuid: "7b916250fc3342458c430e8d1357d03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } } }
I20260806 22:40:11.910599 4627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b916250fc3342458c430e8d1357d03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b916250fc3342458c430e8d1357d03f" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44199 } } }
I20260806 22:40:11.910785 4627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:11.910714 4626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:11.911027 4640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:11.911607 4640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:11.913380 4640 catalog_manager.cc:1347] Generated new cluster ID: 1cdb107ad71b46d483ff4629308201d3
I20260806 22:40:11.913434 4640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:11 dist-test-slave-gjrz krb5kdc[4486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:11.918919 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.908729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.0.186.254:44199)
0806 22:40:11.909372 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:11.909380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:11.909405 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:40:11.909515 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:11.909520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:11.909559 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:11.909674 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:11.910452 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.911966 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.912994 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.913448 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.913557 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:11.915745 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:11.915749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:11.915749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:11.915760 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:11.917213 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:11.917528 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:11.917531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:11.917532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:11.917636 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:11.917810 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:11.917811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:11.917811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:11.918034 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:11.918631 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:11.918687 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:40:11.918788 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":561,"threads_started":1}
I20260806 22:40:11.949724 4640 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:11.950268 4640 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:11.952754 4640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b916250fc3342458c430e8d1357d03f: Generated new TSK 0
I20260806 22:40:11.952929 4640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:11.990706 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:12.073210 4658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:12.073462 4658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:12.073496 4658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:12.075156 4658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:12.075210 4658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:12.075246 4658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:12.077478 4658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:12.077734 4658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:12.078004 4658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:12.084470 4671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.084472 4668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.084487 4667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:12.084967 4658 server_base.cc:1034] running on GCE node
I20260806 22:40:12.085192 4658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:12.085412 4658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:12.086580 4658 hybrid_clock.cc:648] HybridClock initialized: now 1786056012086559 us; error 29 us; skew 500 ppm
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:12.089790 4658 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:12.090378 4658 webserver.cc:460] Webserver started at http://127.0.186.193:44683/ using document root <none> and password file <none>
I20260806 22:40:12.090660 4658 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:12.090730 4658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:12.090884 4658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:12.092355 4658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "90682635c6444712b981813d15fe8baa"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "c2c80fde67b49356a1b1cd0828936aca"
server_key_iv: "669b0e2be6af81007a0640928364cc08"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.092852 4658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "90682635c6444712b981813d15fe8baa"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "c2c80fde67b49356a1b1cd0828936aca"
server_key_iv: "669b0e2be6af81007a0640928364cc08"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.094805 4658 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:40:12.095624 4687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.095791 4658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.095880 4658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "90682635c6444712b981813d15fe8baa"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "c2c80fde67b49356a1b1cd0828936aca"
server_key_iv: "669b0e2be6af81007a0640928364cc08"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.095957 4658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:12.111841 4658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:12.113870 4658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:12.114004 4658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:12.114238 4658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:12.114543 4658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:12.114578 4658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.114609 4658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:12.114636 4658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.120278 4658 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:46815
I20260806 22:40:12.120334 4868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:46815 every 8 connection(s)
I20260806 22:40:12.120638 4658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:12.125559 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 4658
I20260806 22:40:12.125643 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:12.125784 747 external_mini_cluster.cc:1428] Setting key e8e225f44d9eb97c8b9be72202b940e0
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:12.130040 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.121302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:50675 (local address 127.0.186.254:44199)
0806 22:40:12.121353 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:12.121356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:12.121523 (+ 167us) server_negotiation.cc:407] Connection header received
0806 22:40:12.122584 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:12.122585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:12.122601 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:12.122654 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:12.123241 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.123772 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.124848 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.125074 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.125130 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.127357 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:12.127359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:12.127359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:12.127366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:12.128657 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.129098 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.129098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.129099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.129145 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.129393 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.129394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.129394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.129520 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:12.129600 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:12.129859 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 22:40:12.129951 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:12.130111 4871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.121270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44199 (local address 127.0.186.193:50675)
0806 22:40:12.121405 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.121440 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:12.122460 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:12.122724 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:12.122728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:12.122965 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:12.123149 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.123154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.123861 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.123862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.124760 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.124764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.125146 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.125147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.125259 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.125702 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:12.125713 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:12.127210 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:12.128735 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.128742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.128744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.129025 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.129215 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.129216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.129216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.129331 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.129606 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:12.129613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:12.129730 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 22:40:12.129824 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:40:12.129945 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260806 22:40:12.130592 4869 heartbeater.cc:347] Connected to a master server at 127.0.186.254:44199
I20260806 22:40:12.130694 4869 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:12.130892 4869 heartbeater.cc:511] Master 127.0.186.254:44199 requested a full tablet report, sending...
I20260806 22:40:12.131625 4556 ts_manager.cc:194] Registered new tserver with Master: 90682635c6444712b981813d15fe8baa (127.0.186.193:46815)
I20260806 22:40:12.132222 4556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:50675
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:12.150741 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:12.231060 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:12.231249 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:12.231294 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:12.232404 4875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:12.232439 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:12.232465 4875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:12.233932 4875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:12.234118 4875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:12.234297 4875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:12.238906 4882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.238901 4886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.238901 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:12.238994 4875 server_base.cc:1034] running on GCE node
I20260806 22:40:12.239372 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:12.239601 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:12.240734 4875 hybrid_clock.cc:648] HybridClock initialized: now 1786056012240722 us; error 29 us; skew 500 ppm
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:12.243062 4875 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:12.243510 4875 webserver.cc:460] Webserver started at http://127.0.186.194:37045/ using document root <none> and password file <none>
I20260806 22:40:12.243712 4875 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:12.243759 4875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:12.243876 4875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:12.244890 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "684d61087ae44e299b119f94af210dd0"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "7937da5306e495f52f2fb00d5ce5badc"
server_key_iv: "00a1b0a9c7f7a38b0d0b1ebe533e24ba"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.245285 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "684d61087ae44e299b119f94af210dd0"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "7937da5306e495f52f2fb00d5ce5badc"
server_key_iv: "00a1b0a9c7f7a38b0d0b1ebe533e24ba"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.246490 4875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:12.247118 4897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.247287 4875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:12.247377 4875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "684d61087ae44e299b119f94af210dd0"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "7937da5306e495f52f2fb00d5ce5badc"
server_key_iv: "00a1b0a9c7f7a38b0d0b1ebe533e24ba"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.247439 4875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:12.263825 4875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:12.265460 4875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:12.265600 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:12.265798 4875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:12.266081 4875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:12.266114 4875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.266141 4875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:12.266157 4875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.271463 4875 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:46863
I20260806 22:40:12.271502 5059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:46863 every 8 connection(s)
I20260806 22:40:12.271790 4875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:12.275149 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 4875
I20260806 22:40:12.275238 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:12.275424 747 external_mini_cluster.cc:1428] Setting key 531df0792ccebfdf05059a2776cf90f6
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:12.280791 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.272438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:39921 (local address 127.0.186.254:44199)
0806 22:40:12.272494 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:12.272497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:12.272682 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:40:12.273562 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:12.273563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:12.273571 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:12.273624 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:12.274180 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.274675 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.275715 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.275995 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.276062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.278304 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:12.278306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:12.278306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:12.278314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:12.279550 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.279910 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.279911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.279911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.279967 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.280192 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.280193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.280193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.280306 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:12.280361 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:12.280614 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:40:12.280702 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:40:12.280989 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.272401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44199 (local address 127.0.186.194:39921)
0806 22:40:12.272566 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.272607 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:12.273483 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:12.273681 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:12.273685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:12.273921 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:12.274111 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.274115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.274734 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.274735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.275631 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.275634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.276079 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.276080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.276220 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.276640 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:12.276650 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:12.278218 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:12.279622 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.279630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.279632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.279845 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.280020 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.280021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.280021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.280135 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.280360 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:12.280366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:12.280494 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 22:40:12.280581 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:40:12.280729 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 22:40:12.281523 5060 heartbeater.cc:347] Connected to a master server at 127.0.186.254:44199
I20260806 22:40:12.281610 5060 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:12.281770 5060 heartbeater.cc:511] Master 127.0.186.254:44199 requested a full tablet report, sending...
I20260806 22:40:12.282161 4556 ts_manager.cc:194] Registered new tserver with Master: 684d61087ae44e299b119f94af210dd0 (127.0.186.194:46863)
I20260806 22:40:12.282665 4556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:39921
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:12.301453 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:12.382511 5066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:12.382704 5066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:12.382726 5066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:12.383834 5066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:12.383868 5066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:12.383893 5066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:12.385341 5066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:12.385535 5066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:12.385703 5066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:12.390298 5079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.390306 5074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:12.390301 5077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:12.390611 5066 server_base.cc:1034] running on GCE node
I20260806 22:40:12.390812 5066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:12.391044 5066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:12.392190 5066 hybrid_clock.cc:648] HybridClock initialized: now 1786056012392170 us; error 35 us; skew 500 ppm
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:12.394471 5066 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:12.394886 5066 webserver.cc:460] Webserver started at http://127.0.186.195:34673/ using document root <none> and password file <none>
I20260806 22:40:12.395099 5066 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:12.395147 5066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:12.395298 5066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:12.396327 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "310a6e04533e4e10bfa7f7dbf81aa81a"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "71f15a48191d44d55dea0f19111f2579"
server_key_iv: "1fe254e18e56d19ba007d62cb9ec1971"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.396656 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "310a6e04533e4e10bfa7f7dbf81aa81a"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "71f15a48191d44d55dea0f19111f2579"
server_key_iv: "1fe254e18e56d19ba007d62cb9ec1971"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.397858 5066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:12.398514 5089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.398659 5066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:12.398729 5066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "310a6e04533e4e10bfa7f7dbf81aa81a"
format_stamp: "Formatted at 2026-08-06 22:40:12 on dist-test-slave-gjrz"
server_key: "71f15a48191d44d55dea0f19111f2579"
server_key_iv: "1fe254e18e56d19ba007d62cb9ec1971"
server_key_version: "encryptionkey@0"
I20260806 22:40:12.398794 5066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:12.431999 5066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:12.434046 5066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:12.434181 5066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:12.434388 5066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:12.434697 5066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:12.434729 5066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.434762 5066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:12.434783 5066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:12.440551 5066 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:41869
I20260806 22:40:12.440595 5251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:41869 every 8 connection(s)
I20260806 22:40:12.440889 5066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[4486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:12.446866 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 5066
I20260806 22:40:12.446946 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:12.447151 747 external_mini_cluster.cc:1428] Setting key 5bdb706233376eff77c025333b350f53
I20260806 22:40:12.449472 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.441577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:37565 (local address 127.0.186.254:44199)
0806 22:40:12.441642 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:12.441644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:12.441802 (+ 158us) server_negotiation.cc:407] Connection header received
0806 22:40:12.442743 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:12.442745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:12.442757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:12.442807 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:12.443447 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.443959 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.444898 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.445097 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.445153 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.447046 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:12.447047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:12.447048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:12.447056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:12.448433 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.448783 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.448784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.448784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.448828 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:12.449003 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:12.449004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:12.449004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:12.449095 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:12.449139 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:12.449352 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:40:12.449411 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:40:12.449652 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.441517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44199 (local address 127.0.186.195:37565)
0806 22:40:12.441685 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.441728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:12.442639 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:12.442880 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:12.442884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:12.443127 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:12.443356 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.443361 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.444038 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.444040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.444817 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:12.444820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:12.445165 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:12.445166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:12.445297 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:12.445671 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:12.445679 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:12.446957 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:12.448496 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.448502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.448504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.448727 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.448871 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:12.448871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:12.448872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:12.448963 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:12.449138 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:12.449141 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:12.449253 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 22:40:12.449325 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:40:12.449440 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260806 22:40:12.450054 5252 heartbeater.cc:347] Connected to a master server at 127.0.186.254:44199
I20260806 22:40:12.450142 5252 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:12.450306 5252 heartbeater.cc:511] Master 127.0.186.254:44199 requested a full tablet report, sending...
I20260806 22:40:12.450640 4556 ts_manager.cc:194] Registered new tserver with Master: 310a6e04533e4e10bfa7f7dbf81aa81a (127.0.186.195:41869)
I20260806 22:40:12.451038 4556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:37565
I20260806 22:40:12.458721 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:40:12.471092 4625 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:12.470239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.0.186.254:44199)
0806 22:40:12.470286 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:12.470288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:12.470402 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:40:12.470469 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:12.470471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:12.470480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:12.470519 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:12.471029 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60702: BlockingRecv error: recv got EOF from 127.0.0.1:60702 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:12.471848 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 4658
I20260806 22:40:12.491344 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 4875
I20260806 22:40:12.501039 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 5066
I20260806 22:40:12.512148 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 4508
2026-08-06T22:40:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1986 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[5273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[5273](info): set up 2 sockets
Aug 06 22:40:12 dist-test-slave-gjrz krb5kdc[5273](info): commencing operation
krb5kdc: starting...
W20260806 22:40:13.636827 747 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:13.749408 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42083
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:13.829833 5295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:13.830037 5295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:13.830058 5295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:13.831200 5295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:13.831240 5295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:13.831254 5295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:13.831290 5295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:13.831314 5295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:13.832813 5295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:13.833048 5295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:13.833272 5295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:13.837900 5306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:13.837896 5303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:13.837946 5295 server_base.cc:1034] running on GCE node
W20260806 22:40:13.837901 5302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:13.838337 5295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:13.838621 5295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:13.839763 5295 hybrid_clock.cc:648] HybridClock initialized: now 1786056013839738 us; error 40 us; skew 500 ppm
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056013, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:13.844810 5295 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:13.845273 5295 webserver.cc:460] Webserver started at http://127.0.186.254:42307/ using document root <none> and password file <none>
I20260806 22:40:13.845487 5295 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:13.845532 5295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:13.845661 5295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:13.846671 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "28110c00fce2459eb4485b30fc64d1fc"
format_stamp: "Formatted at 2026-08-06 22:40:13 on dist-test-slave-gjrz"
server_key: "198ffa6d000f0e66dd3f7c64e84cecea"
server_key_iv: "17d9601a9d8896c217ce2563ad8f4261"
server_key_version: "encryptionkey@0"
I20260806 22:40:13.847012 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "28110c00fce2459eb4485b30fc64d1fc"
format_stamp: "Formatted at 2026-08-06 22:40:13 on dist-test-slave-gjrz"
server_key: "198ffa6d000f0e66dd3f7c64e84cecea"
server_key_iv: "17d9601a9d8896c217ce2563ad8f4261"
server_key_version: "encryptionkey@0"
I20260806 22:40:13.848281 5295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:13.848958 5315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:13.849107 5295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:13.849186 5295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "28110c00fce2459eb4485b30fc64d1fc"
format_stamp: "Formatted at 2026-08-06 22:40:13 on dist-test-slave-gjrz"
server_key: "198ffa6d000f0e66dd3f7c64e84cecea"
server_key_iv: "17d9601a9d8896c217ce2563ad8f4261"
server_key_version: "encryptionkey@0"
I20260806 22:40:13.849257 5295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:13.864168 5295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:13.865764 5295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:13.865888 5295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:13.869707 5295 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42083
I20260806 22:40:13.869740 5387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42083 every 8 connection(s)
I20260806 22:40:13.870049 5295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:13.870872 5391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:13.873126 5391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc: Bootstrap starting.
I20260806 22:40:13.873703 5391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:13.873819 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 5295
I20260806 22:40:13.873916 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:13.873996 5391 log.cc:824] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:13.874080 747 external_mini_cluster.cc:1428] Setting key 33a5d0472a25244cf715564ec266c6c0
I20260806 22:40:13.874699 5391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc: No bootstrap required, opened a new log
I20260806 22:40:13.876772 5391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } }
I20260806 22:40:13.876905 5391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:13.876936 5391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28110c00fce2459eb4485b30fc64d1fc, State: Initialized, Role: FOLLOWER
I20260806 22:40:13.877050 5391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } }
I20260806 22:40:13.877122 5391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:13.877152 5391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:13.877198 5391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:13.877704 5391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } }
I20260806 22:40:13.877799 5391 leader_election.cc:302] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28110c00fce2459eb4485b30fc64d1fc; no voters:
I20260806 22:40:13.877947 5391 leader_election.cc:288] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:13.877992 5398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:13.878110 5398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [term 1 LEADER]: Becoming Leader. State: Replica: 28110c00fce2459eb4485b30fc64d1fc, State: Running, Role: LEADER
I20260806 22:40:13.878198 5391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:13.878192 5398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } }
I20260806 22:40:13.878978 5402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28110c00fce2459eb4485b30fc64d1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } } }
I20260806 22:40:13.879113 5400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28110c00fce2459eb4485b30fc64d1fc. Latest consensus state: current_term: 1 leader_uuid: "28110c00fce2459eb4485b30fc64d1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28110c00fce2459eb4485b30fc64d1fc" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42083 } } }
I20260806 22:40:13.879148 5400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:13.879072 5402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:13.879366 5413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:13.879870 5413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:13.881630 5413 catalog_manager.cc:1347] Generated new cluster ID: f2af7b0e27c94391b41911be822ab0bf
I20260806 22:40:13.881682 5413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:13.883926 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:13.874496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.0.186.254:42083)
0806 22:40:13.874675 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:13.874684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:13.874715 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:40:13.875155 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:13.875161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:13.875207 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:13.875347 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:13.875958 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:13.876817 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:13.877642 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:13.878152 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:13.878543 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:13.880973 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:13.880977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:13.880977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:13.880988 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:13.882505 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:13.882829 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:13.882832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:13.882832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:13.882916 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:13.883090 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:13.883091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:13.883091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:13.883364 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:13.883530 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:13.883604 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:40:13.883734 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260806 22:40:13.888051 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:13.905998 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:13.901985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.0.186.254:42083)
0806 22:40:13.902031 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:13.902033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:13.902174 (+ 141us) server_negotiation.cc:407] Connection header received
0806 22:40:13.902240 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:13.902240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:13.902252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:13.902299 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:13.902695 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:13.903469 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:13.904507 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:13.904815 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:13.904870 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:13.905538 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:13.905540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:13.905540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:13.905752 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:13.905770 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:13.905777 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:13.905832 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:13.905894 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:40:13.905921 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:13.906710 5413 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:13.907222 5413 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:13.910914 5413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28110c00fce2459eb4485b30fc64d1fc: Generated new TSK 0
I20260806 22:40:13.911075 5413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:13.917426 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 5295
2026-08-06T22:40:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1406 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5436](info): set up 2 sockets
Aug 06 22:40:13 dist-test-slave-gjrz krb5kdc[5436](info): commencing operation
krb5kdc: starting...
W20260806 22:40:15.055047 747 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:15.167837 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42103
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:15.249456 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:15.249660 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:15.249691 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:15.250828 5461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:15.250869 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:15.250888 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:15.250905 5461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:15.250921 5461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:15.252431 5461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:15.252671 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:15.252872 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:15.257375 5468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.257391 5469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.257579 5472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:15.257591 5461 server_base.cc:1034] running on GCE node
I20260806 22:40:15.257815 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:15.258081 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:15.259243 5461 hybrid_clock.cc:648] HybridClock initialized: now 1786056015259227 us; error 29 us; skew 500 ppm
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:15.261621 5461 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:15.262046 5461 webserver.cc:460] Webserver started at http://127.0.186.254:38585/ using document root <none> and password file <none>
I20260806 22:40:15.262218 5461 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:15.262255 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:15.262343 5461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:15.263413 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "4f329587f6344a19b60dc9dbe0993b58"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "838832489880d276f6c1f399aad2a3ec"
server_key_iv: "66d8da28c2e5a986d1b70e4253f1a329"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.263715 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "4f329587f6344a19b60dc9dbe0993b58"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "838832489880d276f6c1f399aad2a3ec"
server_key_iv: "66d8da28c2e5a986d1b70e4253f1a329"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.264930 5461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:15.265638 5482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.265821 5461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:15.265918 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "4f329587f6344a19b60dc9dbe0993b58"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "838832489880d276f6c1f399aad2a3ec"
server_key_iv: "66d8da28c2e5a986d1b70e4253f1a329"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.265980 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:15.294584 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:15.296355 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:15.296489 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:15.300634 5461 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42103
I20260806 22:40:15.300674 5553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42103 every 8 connection(s)
I20260806 22:40:15.300997 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:15.302260 5554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:15.302266 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 5461
I20260806 22:40:15.302352 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:15.302562 747 external_mini_cluster.cc:1428] Setting key a9a21862b2aaf85cdcebd9b380f889c6
I20260806 22:40:15.305527 5554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58: Bootstrap starting.
I20260806 22:40:15.306167 5554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:15.306470 5554 log.cc:824] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:15.307200 5554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58: No bootstrap required, opened a new log
I20260806 22:40:15.309105 5554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } }
I20260806 22:40:15.309276 5554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:15.309309 5554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f329587f6344a19b60dc9dbe0993b58, State: Initialized, Role: FOLLOWER
I20260806 22:40:15.309449 5554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } }
I20260806 22:40:15.309540 5554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:15.309573 5554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:15.309623 5554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:15.310141 5554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } }
I20260806 22:40:15.310253 5554 leader_election.cc:302] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f329587f6344a19b60dc9dbe0993b58; no voters:
I20260806 22:40:15.310416 5554 leader_election.cc:288] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:15.310472 5562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:15.310611 5562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [term 1 LEADER]: Becoming Leader. State: Replica: 4f329587f6344a19b60dc9dbe0993b58, State: Running, Role: LEADER
I20260806 22:40:15.310683 5554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:15.310757 5562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } }
I20260806 22:40:15.311201 5563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f329587f6344a19b60dc9dbe0993b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } } }
I20260806 22:40:15.311285 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f329587f6344a19b60dc9dbe0993b58. Latest consensus state: current_term: 1 leader_uuid: "4f329587f6344a19b60dc9dbe0993b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f329587f6344a19b60dc9dbe0993b58" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42103 } } }
I20260806 22:40:15.311322 5563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:15.311355 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:15.311517 5575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:15.312077 5575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:15.313650 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.303077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.0.186.254:42103)
0806 22:40:15.303228 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:15.303236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:15.303284 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:40:15.303777 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:15.303789 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:15.303840 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:15.303955 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:15.304785 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.305905 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.306883 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.307324 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.307427 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.309824 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:15.309828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:15.309829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:15.309840 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:15.312126 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.312478 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.312481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.312482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.312598 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.312833 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.312834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.312834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.313093 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:15.313193 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:15.313340 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:40:15.313442 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260806 22:40:15.316212 5575 catalog_manager.cc:1347] Generated new cluster ID: 412db8ab6e4140d88c9598bff2e7f0d6
I20260806 22:40:15.316263 5575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:15.345580 5575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:15.346117 5575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:15.354643 5575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f329587f6344a19b60dc9dbe0993b58: Generated new TSK 0
I20260806 22:40:15.354804 5575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:15.387935 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:15.468964 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:15.469142 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:15.469164 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:15.470326 5593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:15.470371 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:15.470400 5593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:15.471984 5593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:15.472211 5593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:15.472416 5593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:15.477022 5602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:15.477210 5593 server_base.cc:1034] running on GCE node
W20260806 22:40:15.477016 5604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.477017 5607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:15.477448 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:15.477660 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:15.478794 5593 hybrid_clock.cc:648] HybridClock initialized: now 1786056015478789 us; error 28 us; skew 500 ppm
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:15.481108 5593 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:15.481489 5593 webserver.cc:460] Webserver started at http://127.0.186.193:46487/ using document root <none> and password file <none>
I20260806 22:40:15.481694 5593 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:15.481742 5593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:15.481861 5593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:15.482846 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "566c2a94da3d4513a014b1083ea67a9b"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "02b01423319bc4744361e5b0481f3b51"
server_key_iv: "b72ded5b4e1a4e3d4046fec04362ef95"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.483162 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "566c2a94da3d4513a014b1083ea67a9b"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "02b01423319bc4744361e5b0481f3b51"
server_key_iv: "b72ded5b4e1a4e3d4046fec04362ef95"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.484364 5593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:15.484997 5618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.485157 5593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:15.485237 5593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "566c2a94da3d4513a014b1083ea67a9b"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "02b01423319bc4744361e5b0481f3b51"
server_key_iv: "b72ded5b4e1a4e3d4046fec04362ef95"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.485298 5593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:15.497504 5593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:15.499814 5593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:15.499917 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:15.500077 5593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:15.500344 5593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:15.500376 5593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.500402 5593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:15.500419 5593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.505836 5593 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:33161
I20260806 22:40:15.505898 5800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:33161 every 8 connection(s)
I20260806 22:40:15.506147 5593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:15.512719 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 5593
I20260806 22:40:15.512815 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:15.512985 747 external_mini_cluster.cc:1428] Setting key 289a3e091bb1ee5e694bcf9a6235117b
I20260806 22:40:15.514950 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.506787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:57439 (local address 127.0.186.254:42103)
0806 22:40:15.506845 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:15.506847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:15.507057 (+ 210us) server_negotiation.cc:407] Connection header received
0806 22:40:15.507575 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:15.507576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:15.507591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:15.507643 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:15.508527 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.509028 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.509990 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.510204 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.510260 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.512295 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:15.512297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:15.512297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:15.512304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:15.513732 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.514084 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.514085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.514086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.514148 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.514339 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.514340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.514340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.514478 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:15.514540 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:15.514773 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:40:15.514861 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:15.515002 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.506765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42103 (local address 127.0.186.193:57439)
0806 22:40:15.506949 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.506982 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:15.507481 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:15.507705 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:15.507710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:15.508243 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:15.508432 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.508436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.509109 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.509110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.509904 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.509907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.510274 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.510274 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.510401 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.510831 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:15.510867 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:15.512209 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:15.513806 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.513812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.513814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.514015 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.514205 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.514205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.514205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.514288 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.514534 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:15.514538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:15.514658 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 22:40:15.514739 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:40:15.514854 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260806 22:40:15.515869 5802 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42103
I20260806 22:40:15.515959 5802 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:15.516134 5802 heartbeater.cc:511] Master 127.0.186.254:42103 requested a full tablet report, sending...
I20260806 22:40:15.516640 5506 ts_manager.cc:194] Registered new tserver with Master: 566c2a94da3d4513a014b1083ea67a9b (127.0.186.193:33161)
I20260806 22:40:15.517203 5506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:57439
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:15.537817 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:15.616808 5812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:15.616997 5812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:15.617018 5812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:15.618074 5812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:15.618106 5812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:15.618130 5812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:15.619501 5812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:15.619690 5812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:15.619935 5812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:15.624821 5827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.624869 5823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.624835 5820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:15.625006 5812 server_base.cc:1034] running on GCE node
I20260806 22:40:15.625186 5812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:15.625371 5812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:15.626507 5812 hybrid_clock.cc:648] HybridClock initialized: now 1786056015626499 us; error 30 us; skew 500 ppm
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:15.628849 5812 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:15.629245 5812 webserver.cc:460] Webserver started at http://127.0.186.194:38469/ using document root <none> and password file <none>
I20260806 22:40:15.629460 5812 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:15.629508 5812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:15.629628 5812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:15.630589 5812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "5a0e55134f5d46aa82e98ab3a79c4e7e"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "dab01621a815126e0f7ef05479c003fb"
server_key_iv: "1d2ad35745ac36570cd945499f3d3028"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.630910 5812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "5a0e55134f5d46aa82e98ab3a79c4e7e"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "dab01621a815126e0f7ef05479c003fb"
server_key_iv: "1d2ad35745ac36570cd945499f3d3028"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.632184 5812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:15.632844 5838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.632993 5812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:15.633071 5812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "5a0e55134f5d46aa82e98ab3a79c4e7e"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "dab01621a815126e0f7ef05479c003fb"
server_key_iv: "1d2ad35745ac36570cd945499f3d3028"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.633145 5812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:15.641757 5812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:15.643392 5812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:15.643517 5812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:15.643707 5812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:15.643985 5812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:15.644027 5812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.644052 5812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:15.644076 5812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.649576 5812 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:45845
I20260806 22:40:15.649648 6010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:45845 every 8 connection(s)
I20260806 22:40:15.649899 5812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:15.652182 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 5812
I20260806 22:40:15.652287 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:15.652460 747 external_mini_cluster.cc:1428] Setting key f09a3c0b823f38442554da7e53ea29d1
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:15.658504 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.650537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:58839 (local address 127.0.186.254:42103)
0806 22:40:15.650598 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:15.650600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:15.650745 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:40:15.651181 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:15.651182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:15.651192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:15.651246 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:15.652121 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.652694 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.653623 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.653823 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.653878 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.656118 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:15.656120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:15.656120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:15.656128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:15.657256 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.657627 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.657628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.657629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.657683 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.657901 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.657902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.657902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.658011 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:15.658068 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:15.658315 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 22:40:15.658415 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:40:15.658645 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.650488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42103 (local address 127.0.186.194:58839)
0806 22:40:15.650633 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.650671 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:15.651107 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:15.651328 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:15.651331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:15.651839 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:15.652012 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.652016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.652774 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.652776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.653533 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.653535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.653890 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.653891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.653999 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.654444 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:15.654453 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:15.656027 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:15.657331 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.657338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.657340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.657557 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.657739 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.657739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.657740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.657844 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.658066 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:15.658074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:15.658200 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:40:15.658286 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:40:15.658432 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":87,"threads_started":1}
I20260806 22:40:15.659060 6012 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42103
I20260806 22:40:15.659142 6012 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:15.659334 6012 heartbeater.cc:511] Master 127.0.186.254:42103 requested a full tablet report, sending...
I20260806 22:40:15.659731 5506 ts_manager.cc:194] Registered new tserver with Master: 5a0e55134f5d46aa82e98ab3a79c4e7e (127.0.186.194:45845)
I20260806 22:40:15.660225 5506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:58839
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:15.675614 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:15.754369 6019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:15.754566 6019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:15.754588 6019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:15.755772 6019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:15.755816 6019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:15.755843 6019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:15.757288 6019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:15.757503 6019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:15.757721 6019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:15.762404 6026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.762418 6025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:15.762410 6030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:15.762674 6019 server_base.cc:1034] running on GCE node
I20260806 22:40:15.762857 6019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:15.763065 6019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:15.764233 6019 hybrid_clock.cc:648] HybridClock initialized: now 1786056015764208 us; error 28 us; skew 500 ppm
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:15.766635 6019 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:15.767012 6019 webserver.cc:460] Webserver started at http://127.0.186.195:46389/ using document root <none> and password file <none>
I20260806 22:40:15.767201 6019 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:15.767247 6019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:15.767388 6019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:15.768381 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "8d7122f29c4144bda2ad3eb88dfe3ffd"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "97c3721095484b0c2f389f88df3c9766"
server_key_iv: "b445b5222c759cacc6dc82c6165d1e7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.768730 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "8d7122f29c4144bda2ad3eb88dfe3ffd"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "97c3721095484b0c2f389f88df3c9766"
server_key_iv: "b445b5222c759cacc6dc82c6165d1e7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.770030 6019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:15.770753 6047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.770927 6019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:15.771018 6019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "8d7122f29c4144bda2ad3eb88dfe3ffd"
format_stamp: "Formatted at 2026-08-06 22:40:15 on dist-test-slave-gjrz"
server_key: "97c3721095484b0c2f389f88df3c9766"
server_key_iv: "b445b5222c759cacc6dc82c6165d1e7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:15.771078 6019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:15.788740 6019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:15.790689 6019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:15.790827 6019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:15.791028 6019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:15.791352 6019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:15.791388 6019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.791416 6019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:15.791447 6019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:15.796855 6019 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:43225
I20260806 22:40:15.796916 6213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:43225 every 8 connection(s)
I20260806 22:40:15.797183 6019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:15.799916 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 6019
I20260806 22:40:15.800004 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:15.800196 747 external_mini_cluster.cc:1428] Setting key bde9583abf6261260512b5a2f516bd4c
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:15.805650 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.797801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:53937 (local address 127.0.186.254:42103)
0806 22:40:15.797876 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:15.797878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:15.798060 (+ 182us) server_negotiation.cc:407] Connection header received
0806 22:40:15.798499 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:15.798500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:15.798511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:15.798557 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:15.799455 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.799958 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.801277 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.801477 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.801531 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.803506 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:15.803507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:15.803508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:15.803515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:15.804600 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.804927 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.804928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.804928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.804975 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.805155 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.805155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.805155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.805260 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:15.805310 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:15.805519 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:40:15.805586 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:40:15.805776 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.797781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42103 (local address 127.0.186.195:53937)
0806 22:40:15.797936 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.797969 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:15.798413 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:15.798616 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:15.798618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:15.799149 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:15.799354 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.799359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.800070 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.800072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.801199 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:15.801203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.801543 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.801544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:15.801644 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.802067 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:15.802075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:15.803407 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:15.804667 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.804672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.804674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.804873 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.805020 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:15.805021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:15.805021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:15.805109 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:15.805307 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:15.805310 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:15.805411 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 22:40:15.805494 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:40:15.805607 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260806 22:40:15.806216 6214 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42103
I20260806 22:40:15.806304 6214 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:15.806469 6214 heartbeater.cc:511] Master 127.0.186.254:42103 requested a full tablet report, sending...
I20260806 22:40:15.806843 5506 ts_manager.cc:194] Registered new tserver with Master: 8d7122f29c4144bda2ad3eb88dfe3ffd (127.0.186.195:43225)
I20260806 22:40:15.807229 5506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:53937
I20260806 22:40:15.811807 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056015, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:15.834980 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.828114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.0.186.254:42103)
0806 22:40:15.828168 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:15.828170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:15.828267 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:40:15.828310 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:15.828311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:15.828319 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:15.828366 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:15.828919 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.829414 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.830154 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:15.830342 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:15.830401 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:15.832648 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:15.832650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:15.832650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:15.832656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:15.833904 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.834227 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.834228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.834235 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.834289 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:15.834459 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:15.834459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:15.834460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:15.834598 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:15.834686 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:15.834803 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:40:15.834893 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 22:40:15.835201 5506 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52516
I20260806 22:40:15.835672 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 5593
I20260806 22:40:15.851732 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 5812
I20260806 22:40:15.862679 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 6019
I20260806 22:40:15.873210 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 5461
2026-08-06T22:40:15Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1949 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[6236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[6236](info): set up 2 sockets
Aug 06 22:40:15 dist-test-slave-gjrz krb5kdc[6236](info): commencing operation
krb5kdc: starting...
W20260806 22:40:16.993639 747 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:17.110239 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40571
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:40571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:17.190217 6267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:17.190407 6267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:17.190428 6267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:17.191576 6267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:17.191620 6267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:17.191633 6267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:17.191645 6267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:17.191656 6267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:17.193204 6267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:40571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:17.193454 6267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:17.193674 6267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:17.198369 6279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.198359 6281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.198357 6276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.198439 6267 server_base.cc:1034] running on GCE node
I20260806 22:40:17.198808 6267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:17.199126 6267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:17.200265 6267 hybrid_clock.cc:648] HybridClock initialized: now 1786056017200253 us; error 38 us; skew 500 ppm
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:17.202683 6267 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:17.203220 6267 webserver.cc:460] Webserver started at http://127.0.186.254:41011/ using document root <none> and password file <none>
I20260806 22:40:17.203444 6267 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:17.203495 6267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:17.203621 6267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:17.204613 6267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "9df07c4b055540a6b10352390a980eb5"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "45a42ccd25eb8e5b54c2a7fb6348e9c6"
server_key_iv: "f565c455b6b0dcb11a42718e248dbdb5"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.204938 6267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "9df07c4b055540a6b10352390a980eb5"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "45a42ccd25eb8e5b54c2a7fb6348e9c6"
server_key_iv: "f565c455b6b0dcb11a42718e248dbdb5"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.206174 6267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:17.206825 6292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.206974 6267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:17.207086 6267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "9df07c4b055540a6b10352390a980eb5"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "45a42ccd25eb8e5b54c2a7fb6348e9c6"
server_key_iv: "f565c455b6b0dcb11a42718e248dbdb5"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.207147 6267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:17.228432 6267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:17.230134 6267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:17.230263 6267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:17.234730 6267 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:40571
I20260806 22:40:17.234798 6374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:40571 every 8 connection(s)
I20260806 22:40:17.235236 6267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:17.236255 6377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:17.238467 6377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5: Bootstrap starting.
I20260806 22:40:17.239032 6377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:17.239348 6377 log.cc:824] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:17.239971 6377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5: No bootstrap required, opened a new log
I20260806 22:40:17.241586 6377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } }
I20260806 22:40:17.241724 6377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:17.241744 6377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df07c4b055540a6b10352390a980eb5, State: Initialized, Role: FOLLOWER
I20260806 22:40:17.241897 6377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } }
I20260806 22:40:17.241969 6377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:17.242003 6377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:17.242061 6377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:17.242595 6377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } }
I20260806 22:40:17.242690 6377 leader_election.cc:302] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9df07c4b055540a6b10352390a980eb5; no voters:
I20260806 22:40:17.242842 6377 leader_election.cc:288] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:17.242875 6380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:17.243098 6377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:17.243232 6380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [term 1 LEADER]: Becoming Leader. State: Replica: 9df07c4b055540a6b10352390a980eb5, State: Running, Role: LEADER
I20260806 22:40:17.243364 6380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } }
I20260806 22:40:17.243716 6381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9df07c4b055540a6b10352390a980eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } } }
I20260806 22:40:17.243733 6382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df07c4b055540a6b10352390a980eb5. Latest consensus state: current_term: 1 leader_uuid: "9df07c4b055540a6b10352390a980eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df07c4b055540a6b10352390a980eb5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40571 } } }
I20260806 22:40:17.243803 6381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:17.243830 6382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:17.244016 6391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:17.244568 6391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:17.245067 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 6267
I20260806 22:40:17.245177 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:17.245339 747 external_mini_cluster.cc:1428] Setting key 6f8e06e70fc1a4717ee88dd14962c3ec
I20260806 22:40:17.247026 6391 catalog_manager.cc:1347] Generated new cluster ID: dec530569c254487b98c8c60abdc740a
I20260806 22:40:17.247078 6391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:17.256379 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.245771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.0.186.254:40571)
0806 22:40:17.246114 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.246123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.246154 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:40:17.246244 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.246250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.246290 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.246402 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.247009 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.248454 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.249504 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.250095 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.250181 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.252857 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.252862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.252862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.252876 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.254756 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.255160 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.255163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.255163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.255296 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.255472 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.255473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.255473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.255783 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.255932 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.255998 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.256140 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":61,"threads_started":1}
I20260806 22:40:17.270462 6391 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:17.270951 6391 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:17.274783 6391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9df07c4b055540a6b10352390a980eb5: Generated new TSK 0
I20260806 22:40:17.274928 6391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:17.305826 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:17.386730 6424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:17.386934 6424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:17.386957 6424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:17.388127 6424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:17.388172 6424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:17.388197 6424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:17.389672 6424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:17.389899 6424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:17.390120 6424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:17.395006 6430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.395257 6424 server_base.cc:1034] running on GCE node
W20260806 22:40:17.395126 6431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.395045 6433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.395609 6424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:17.395846 6424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:17.396986 6424 hybrid_clock.cc:648] HybridClock initialized: now 1786056017396970 us; error 28 us; skew 500 ppm
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:17.399298 6424 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:17.399700 6424 webserver.cc:460] Webserver started at http://127.0.186.193:33311/ using document root <none> and password file <none>
I20260806 22:40:17.399923 6424 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:17.399973 6424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:17.400104 6424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:17.401113 6424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "5fd65eb5f1534c8782934749f5e5eaff"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "da45e47ae2ad2de7ae035420fe173f82"
server_key_iv: "787d5c83bbc2762be6c9e0bf20edacbc"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.401445 6424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "5fd65eb5f1534c8782934749f5e5eaff"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "da45e47ae2ad2de7ae035420fe173f82"
server_key_iv: "787d5c83bbc2762be6c9e0bf20edacbc"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.402684 6424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:17.403389 6444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.403533 6424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:17.403609 6424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "5fd65eb5f1534c8782934749f5e5eaff"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "da45e47ae2ad2de7ae035420fe173f82"
server_key_iv: "787d5c83bbc2762be6c9e0bf20edacbc"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.403678 6424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:17.427186 6424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:17.429030 6424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:17.429173 6424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:17.429388 6424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:17.429685 6424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:17.429719 6424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.429749 6424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:17.429771 6424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.440846 6424 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:45539
I20260806 22:40:17.440914 6624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:45539 every 8 connection(s)
I20260806 22:40:17.441295 6424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:17.450016 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.441977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:57335 (local address 127.0.186.254:40571)
0806 22:40:17.442030 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.442032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.442216 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:40:17.443169 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.443171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.443187 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.443238 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.443921 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.444471 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.445490 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.445708 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.445764 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.447687 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.447688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.447689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.447697 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.448769 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.449134 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.449135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.449136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.449189 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.449404 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.449405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.449405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.449518 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.449580 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.449832 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.449925 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:17.450188 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.441954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40571 (local address 127.0.186.193:57335)
0806 22:40:17.442100 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.442138 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:17.443069 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:17.443333 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:17.443337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:17.443613 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:17.443835 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.443841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.444555 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.444556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.445411 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.445413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.445777 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.445778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.445908 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.446284 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:17.446293 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:17.447597 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:17.448839 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.448845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.448847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.449054 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.449244 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.449245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.449245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.449348 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.449572 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:17.449578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:17.449713 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:40:17.449799 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:40:17.449942 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260806 22:40:17.450634 6628 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40571
I20260806 22:40:17.450729 6628 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:17.450901 6628 heartbeater.cc:511] Master 127.0.186.254:40571 requested a full tablet report, sending...
I20260806 22:40:17.451109 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 6424
I20260806 22:40:17.451220 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:17.451447 747 external_mini_cluster.cc:1428] Setting key f06fce50c88707cd84297e0ad43d15a8
I20260806 22:40:17.451570 6316 ts_manager.cc:194] Registered new tserver with Master: 5fd65eb5f1534c8782934749f5e5eaff (127.0.186.193:45539)
I20260806 22:40:17.452310 6316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:57335
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:17.475095 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:17.555858 6636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:17.556061 6636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:17.556094 6636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:17.557210 6636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:17.557251 6636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:17.557286 6636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:17.558724 6636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:17.558954 6636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:17.559152 6636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:17.563925 6647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.563993 6636 server_base.cc:1034] running on GCE node
W20260806 22:40:17.563925 6643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.563979 6642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.564317 6636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:17.564576 6636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:17.565742 6636 hybrid_clock.cc:648] HybridClock initialized: now 1786056017565720 us; error 32 us; skew 500 ppm
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:17.568279 6636 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:17.568713 6636 webserver.cc:460] Webserver started at http://127.0.186.194:40643/ using document root <none> and password file <none>
I20260806 22:40:17.568904 6636 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:17.568953 6636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:17.569077 6636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:17.570078 6636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "7391ab4612ee48daba74ee0d82551039"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "6fb7c5e90d955e0755a8556cc2f62a38"
server_key_iv: "07b6921943c51dad628822b8a0f4057f"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.570417 6636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "7391ab4612ee48daba74ee0d82551039"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "6fb7c5e90d955e0755a8556cc2f62a38"
server_key_iv: "07b6921943c51dad628822b8a0f4057f"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.571660 6636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:17.572300 6662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.572453 6636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:17.572535 6636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "7391ab4612ee48daba74ee0d82551039"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "6fb7c5e90d955e0755a8556cc2f62a38"
server_key_iv: "07b6921943c51dad628822b8a0f4057f"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.572597 6636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:17.584610 6636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:17.586246 6636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:17.586369 6636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:17.586570 6636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:17.586843 6636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:17.586876 6636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.586903 6636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:17.586920 6636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.592531 6636 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43509
I20260806 22:40:17.592592 6838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43509 every 8 connection(s)
I20260806 22:40:17.592864 6636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:17.599175 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 6636
I20260806 22:40:17.599298 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:17.599449 747 external_mini_cluster.cc:1428] Setting key 459defc327bf742d7f827f46e8dc0012
I20260806 22:40:17.601454 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.593486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50541 (local address 127.0.186.254:40571)
0806 22:40:17.593557 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.593560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.593723 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:40:17.594590 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.594591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.594602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.594673 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.595299 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.595800 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.596731 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.596914 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.596958 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.598777 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.598778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.598779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.598785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.600204 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.600577 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.600578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.600578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.600639 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.600853 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.600854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.600854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.600974 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.601030 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.601281 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.601365 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:40:17.601612 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.593445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40571 (local address 127.0.186.194:50541)
0806 22:40:17.593604 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.593643 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:17.594501 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:17.594725 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:17.594729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:17.594965 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:17.595198 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.595205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.595876 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.595877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.596668 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.596670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.596966 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.596967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.597056 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.597438 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:17.597445 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:17.598685 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:17.600278 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.600284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.600287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.600508 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.600693 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.600694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.600694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.600796 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.601030 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:17.601037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:17.601165 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 22:40:17.601251 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:40:17.601412 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260806 22:40:17.602156 6839 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40571
I20260806 22:40:17.602242 6839 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:17.602411 6839 heartbeater.cc:511] Master 127.0.186.254:40571 requested a full tablet report, sending...
I20260806 22:40:17.602774 6316 ts_manager.cc:194] Registered new tserver with Master: 7391ab4612ee48daba74ee0d82551039 (127.0.186.194:43509)
I20260806 22:40:17.603135 6316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:50541
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:17.623317 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:17.701809 6847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:17.702008 6847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:17.702028 6847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:17.703097 6847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:17.703130 6847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:17.703153 6847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:17.704543 6847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:17.704710 6847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:17.704877 6847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:17.709471 6859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.709573 6856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:17.709470 6854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:17.709837 6847 server_base.cc:1034] running on GCE node
I20260806 22:40:17.710031 6847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:17.710237 6847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:17.711364 6847 hybrid_clock.cc:648] HybridClock initialized: now 1786056017711351 us; error 27 us; skew 500 ppm
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:17.713642 6847 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:17.714061 6847 webserver.cc:460] Webserver started at http://127.0.186.195:38053/ using document root <none> and password file <none>
I20260806 22:40:17.714262 6847 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:17.714308 6847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:17.714428 6847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:17.715410 6847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "30f068c96c0a4effa9345403945784fb"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "96910c090103b3a8fd8c81d69d0d8769"
server_key_iv: "e4c5bb884234d480a489c416e393671c"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.715719 6847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "30f068c96c0a4effa9345403945784fb"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "96910c090103b3a8fd8c81d69d0d8769"
server_key_iv: "e4c5bb884234d480a489c416e393671c"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.716899 6847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:17.717543 6869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.717685 6847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:17.717764 6847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "30f068c96c0a4effa9345403945784fb"
format_stamp: "Formatted at 2026-08-06 22:40:17 on dist-test-slave-gjrz"
server_key: "96910c090103b3a8fd8c81d69d0d8769"
server_key_iv: "e4c5bb884234d480a489c416e393671c"
server_key_version: "encryptionkey@0"
I20260806 22:40:17.717820 6847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:17.727837 6847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:17.729398 6847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:17.729519 6847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:17.729708 6847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:17.729974 6847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:17.730006 6847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.730031 6847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:17.730057 6847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:17.735461 6847 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:37011
I20260806 22:40:17.735520 7049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:37011 every 8 connection(s)
I20260806 22:40:17.735793 6847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:17.737490 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 6847
I20260806 22:40:17.737593 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:17.737771 747 external_mini_cluster.cc:1428] Setting key bcbb26232b299982d7a6abfcb727ad43
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:17.744494 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.736436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:58743 (local address 127.0.186.254:40571)
0806 22:40:17.736499 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.736501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.736656 (+ 155us) server_negotiation.cc:407] Connection header received
0806 22:40:17.737504 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.737506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.737519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.737593 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.738334 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.738909 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.739908 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.740116 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.740168 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.741946 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.741948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.741948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.741955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.743237 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.743586 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.743587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.743588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.743652 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.743861 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.743862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.743863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.744000 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.744065 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.744349 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.744430 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:40:17.744633 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.736406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40571 (local address 127.0.186.195:58743)
0806 22:40:17.736546 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.736584 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:17.737399 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:17.737716 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:17.737720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:17.738032 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:17.738251 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.738257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.738980 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.738981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.739830 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:17.739833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.740177 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.740177 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:17.740269 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.740642 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:17.740650 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:17.741858 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:17.743318 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.743323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.743325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.743517 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.743716 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:17.743717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:17.743717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:17.743813 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:17.744074 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:17.744082 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:17.744210 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 22:40:17.744303 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:40:17.744447 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260806 22:40:17.745100 7050 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40571
I20260806 22:40:17.745186 7050 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:17.745344 7050 heartbeater.cc:511] Master 127.0.186.254:40571 requested a full tablet report, sending...
I20260806 22:40:17.745680 6316 ts_manager.cc:194] Registered new tserver with Master: 30f068c96c0a4effa9345403945784fb (127.0.186.195:37011)
I20260806 22:40:17.746019 6316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:58743
I20260806 22:40:17.749223 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:17.771034 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.764394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.0.186.193:45539)
0806 22:40:17.764595 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.764599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.764609 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:17.764634 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.764636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.764649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.764714 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.765190 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.765882 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.766564 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.766862 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.766909 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.768876 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.768884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.768884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.768891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.770163 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.770458 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.770461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.770461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.770510 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.770641 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.770641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.770642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.770757 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.770858 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.770893 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.770954 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[6236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:17.778340 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.772236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.0.186.254:40571)
0806 22:40:17.772290 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.772292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.772300 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:17.772359 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.772360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.772369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.772414 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.772842 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.773317 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.773966 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.774154 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.774205 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.776203 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.776204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.776204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.776211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.777349 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.777641 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.777642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.777642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.777692 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.777852 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.777853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.777853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.777993 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.778087 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.778175 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.778251 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:40:17.778546 6316 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806
I20260806 22:40:17.778846 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 6424
I20260806 22:40:17.789641 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 6636
I20260806 22:40:17.800899 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 6847
I20260806 22:40:17.808395 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 6267
2026-08-06T22:40:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1939 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[7070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[7070](info): set up 2 sockets
Aug 06 22:40:17 dist-test-slave-gjrz krb5kdc[7070](info): commencing operation
krb5kdc: starting...
W20260806 22:40:18.936781 747 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:19.054579 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42821
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:19.134039 7088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.134240 7088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.134262 7088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.135396 7088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:19.135433 7088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.135447 7088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.135457 7088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:19.135468 7088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:19.136955 7088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:19.137189 7088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.137401 7088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:19.142010 7088 server_base.cc:1034] running on GCE node
W20260806 22:40:19.141993 7106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.142068 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.141994 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.142359 7088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.142622 7088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.143762 7088 hybrid_clock.cc:648] HybridClock initialized: now 1786056019143749 us; error 29 us; skew 500 ppm
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:19.150367 7088 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:19.150701 7088 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.150750 7088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.150862 7088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.151948 7088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "4697edd1fdd84163a6633edd267d6a57"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "113632f5ca611b8031ff8f3fefbdb82e"
server_key_iv: "ddeed7448821bb791dd2fb21d07fc98d"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.152274 7088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "4697edd1fdd84163a6633edd267d6a57"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "113632f5ca611b8031ff8f3fefbdb82e"
server_key_iv: "ddeed7448821bb791dd2fb21d07fc98d"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.153816 7088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:19.154459 7113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.154587 7088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:19.154670 7088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "4697edd1fdd84163a6633edd267d6a57"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "113632f5ca611b8031ff8f3fefbdb82e"
server_key_iv: "ddeed7448821bb791dd2fb21d07fc98d"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.154727 7088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.166177 7088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.167814 7088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.167938 7088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.171798 7088 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42821
I20260806 22:40:19.171828 7194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42821 every 8 connection(s)
I20260806 22:40:19.172080 7088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:19.173072 7196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.175305 7196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: Bootstrap starting.
I20260806 22:40:19.175848 7196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.176080 7196 log.cc:824] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.176674 7196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: No bootstrap required, opened a new log
I20260806 22:40:19.178179 7196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } }
I20260806 22:40:19.178311 7196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.178333 7196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4697edd1fdd84163a6633edd267d6a57, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.178442 7196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } }
I20260806 22:40:19.178516 7196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:19.178545 7196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:19.178577 7196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.179116 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 7088
I20260806 22:40:19.179214 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:19.179371 7196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } }
I20260806 22:40:19.179401 747 external_mini_cluster.cc:1428] Setting key 3b1c18dfe04b31aa1bd5a515c5979204
I20260806 22:40:19.179462 7196 leader_election.cc:302] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4697edd1fdd84163a6633edd267d6a57; no voters:
I20260806 22:40:19.179620 7196 leader_election.cc:288] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:19.179669 7204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:19.179822 7204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [term 1 LEADER]: Becoming Leader. State: Replica: 4697edd1fdd84163a6633edd267d6a57, State: Running, Role: LEADER
I20260806 22:40:19.179883 7196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:19.179992 7204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } }
I20260806 22:40:19.180454 7205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4697edd1fdd84163a6633edd267d6a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } } }
I20260806 22:40:19.180449 7207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4697edd1fdd84163a6633edd267d6a57. Latest consensus state: current_term: 1 leader_uuid: "4697edd1fdd84163a6633edd267d6a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4697edd1fdd84163a6633edd267d6a57" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42821 } } }
I20260806 22:40:19.180565 7205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.180572 7207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.181070 7218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:19.181656 7218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:19.183672 7218 catalog_manager.cc:1347] Generated new cluster ID: 383a2719e9c640bea044c49651ec3e0e
I20260806 22:40:19.183727 7218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:19.190035 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.180028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.0.186.254:42821)
0806 22:40:19.180440 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.180450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.180490 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:40:19.180619 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.180626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.180689 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.180835 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.181634 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.182914 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.184014 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.184435 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.184543 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.186615 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.186619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.186619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:19.186631 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.188627 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.188957 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.188961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.188961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.189043 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.189217 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.189218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.189218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.189447 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.189594 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.189702 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.189842 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":82,"threads_started":1}
I20260806 22:40:19.207146 7218 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:19.207701 7218 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:19.211377 7218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: Generated new TSK 0
I20260806 22:40:19.211548 7218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:19.245819 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:19.354684 7234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.354895 7234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.354919 7234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.356076 7234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.356117 7234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.356143 7234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:19.357636 7234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:19.357851 7234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.358062 7234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:19.362874 7239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.362893 7241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.363000 7234 server_base.cc:1034] running on GCE node
W20260806 22:40:19.362874 7245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.363343 7234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.363570 7234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.364709 7234 hybrid_clock.cc:648] HybridClock initialized: now 1786056019364698 us; error 29 us; skew 500 ppm
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:19.367156 7234 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:19.367463 7234 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.367512 7234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.367601 7234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.368569 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "78083cef8822453d874d14af7fad23ce"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "b5289f16816992cb9e5cdbe49eb2fc64"
server_key_iv: "1f07e40de592a261e1bb9eedf24fba09"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.368867 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "78083cef8822453d874d14af7fad23ce"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "b5289f16816992cb9e5cdbe49eb2fc64"
server_key_iv: "1f07e40de592a261e1bb9eedf24fba09"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.370039 7234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:19.370645 7253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.370783 7234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:19.370858 7234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "78083cef8822453d874d14af7fad23ce"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "b5289f16816992cb9e5cdbe49eb2fc64"
server_key_iv: "1f07e40de592a261e1bb9eedf24fba09"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.370923 7234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.381469 7234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.383106 7234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.383220 7234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.383445 7234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:19.383733 7234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:19.383765 7234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.383785 7234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:19.383797 7234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.389120 7234 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:33411
I20260806 22:40:19.389178 7424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:33411 every 8 connection(s)
I20260806 22:40:19.389400 7234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:19.391516 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 7234
I20260806 22:40:19.391619 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:19.391805 747 external_mini_cluster.cc:1428] Setting key 9f02b53cab43b8e1b476f1ceb498d64e
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:19.398588 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.390051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:37297 (local address 127.0.186.254:42821)
0806 22:40:19.390100 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.390101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.390264 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:40:19.390756 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.390757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.390771 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.390823 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.391855 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.392559 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.393839 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.394030 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.394088 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.396096 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.396097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.396098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:19.396106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.397331 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.397696 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.397697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.397698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.397749 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.397997 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.397998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.397998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.398106 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.398165 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.398415 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.398501 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:19.398741 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.390003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42821 (local address 127.0.186.193:37297)
0806 22:40:19.390149 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.390189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.390657 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.390897 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.390901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.391151 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:19.391753 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.391760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.392654 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.392656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.393765 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.393769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.394104 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.394105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.394246 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.394692 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:19.394700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:19.396004 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:19.397406 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.397414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.397416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.397628 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.397803 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.397804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.397804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.397903 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.398164 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:19.398172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:19.398298 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.398383 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.398537 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260806 22:40:19.399125 7426 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42821
I20260806 22:40:19.399205 7426 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:19.399348 7426 heartbeater.cc:511] Master 127.0.186.254:42821 requested a full tablet report, sending...
I20260806 22:40:19.399843 7140 ts_manager.cc:194] Registered new tserver with Master: 78083cef8822453d874d14af7fad23ce (127.0.186.193:33411)
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
I20260806 22:40:19.400502 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:37297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:19.415843 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:19.497231 7434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.497413 7434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.497442 7434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.498546 7434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.498584 7434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.498618 7434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:19.500119 7434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:19.500334 7434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.500526 7434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:19.505205 7445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.505223 7443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.505204 7447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.505402 7434 server_base.cc:1034] running on GCE node
I20260806 22:40:19.505630 7434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.505849 7434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.506994 7434 hybrid_clock.cc:648] HybridClock initialized: now 1786056019506982 us; error 28 us; skew 500 ppm
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:19.509405 7434 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:19.509701 7434 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.509752 7434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.509876 7434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.510845 7434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "adbc97f35c5e49cc8176aacca603adb1"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "2f1cbf106509894eba376ab168ba159e"
server_key_iv: "22d1b546f6e28fde15f8f6909e530937"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.511181 7434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "adbc97f35c5e49cc8176aacca603adb1"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "2f1cbf106509894eba376ab168ba159e"
server_key_iv: "22d1b546f6e28fde15f8f6909e530937"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.512372 7434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:19.513075 7453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.513247 7434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:19.513330 7434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "adbc97f35c5e49cc8176aacca603adb1"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "2f1cbf106509894eba376ab168ba159e"
server_key_iv: "22d1b546f6e28fde15f8f6909e530937"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.513389 7434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.533929 7434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.535805 7434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.535933 7434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.536139 7434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:19.536422 7434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:19.536461 7434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.536489 7434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:19.536510 7434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.541854 7434 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:44583
I20260806 22:40:19.541916 7648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:44583 every 8 connection(s)
I20260806 22:40:19.542152 7434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:19.550384 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 7434
I20260806 22:40:19.550474 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:19.550534 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.542759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:35511 (local address 127.0.186.254:42821)
0806 22:40:19.542811 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.542813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.543035 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:40:19.543549 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.543550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.543560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.543596 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.544424 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.544929 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.545870 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.546076 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.546141 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.548075 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.548077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.548078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:19.548085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.549282 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.549638 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.549639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.549640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.549692 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.549911 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.549912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.549912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.550021 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.550079 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.550347 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.550434 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:19.550666 747 external_mini_cluster.cc:1428] Setting key 0536953a4f23a364901d409b42903fb4
I20260806 22:40:19.550729 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.542736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42821 (local address 127.0.186.194:35511)
0806 22:40:19.542894 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.542928 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.543455 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.543655 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.543658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.543897 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:19.544349 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.544354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.545015 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.545016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.545777 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.545779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.546145 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.546146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.546237 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.546630 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:19.546638 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:19.547987 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:19.549357 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.549363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.549365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.549571 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.549749 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.549749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.549749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.549856 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.550077 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:19.550084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:19.550213 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.550315 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.550519 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260806 22:40:19.551141 7649 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42821
I20260806 22:40:19.551258 7649 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:19.551427 7649 heartbeater.cc:511] Master 127.0.186.254:42821 requested a full tablet report, sending...
I20260806 22:40:19.551817 7140 ts_manager.cc:194] Registered new tserver with Master: adbc97f35c5e49cc8176aacca603adb1 (127.0.186.194:44583)
I20260806 22:40:19.552256 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:35511
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:19.574050 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:19.653856 7656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.654067 7656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.654100 7656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.655223 7656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.655283 7656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.655323 7656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:19.656804 7656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:19.657037 7656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.657243 7656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:19.661983 7664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.662010 7656 server_base.cc:1034] running on GCE node
W20260806 22:40:19.662010 7665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.661983 7668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.662423 7656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.662652 7656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.663793 7656 hybrid_clock.cc:648] HybridClock initialized: now 1786056019663780 us; error 27 us; skew 500 ppm
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:19.666198 7656 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:19.666502 7656 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.666548 7656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.666657 7656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.667654 7656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "f62d295c37d047bd8b38e34c0f1b64e9"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "fbbaaef9abca2059d9e91d5e536b2204"
server_key_iv: "e0314a69a3e60778b7ed709dcb51af75"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.667969 7656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "f62d295c37d047bd8b38e34c0f1b64e9"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "fbbaaef9abca2059d9e91d5e536b2204"
server_key_iv: "e0314a69a3e60778b7ed709dcb51af75"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.669129 7656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:19.669799 7678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.669988 7656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:19.670069 7656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "f62d295c37d047bd8b38e34c0f1b64e9"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "fbbaaef9abca2059d9e91d5e536b2204"
server_key_iv: "e0314a69a3e60778b7ed709dcb51af75"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.670130 7656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.677769 7656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.679749 7656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.679903 7656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.680097 7656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:19.680400 7656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:19.680456 7656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.680492 7656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:19.680516 7656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.685837 7656 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:46707
I20260806 22:40:19.685889 7857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:46707 every 8 connection(s)
I20260806 22:40:19.686121 7656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:19.688395 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 7656
I20260806 22:40:19.688501 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:19.688685 747 external_mini_cluster.cc:1428] Setting key d19084d381e00a73f3c337747941082e
Aug 06 22:40:19 dist-test-slave-gjrz krb5kdc[7070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:19.694741 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.686729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:46965 (local address 127.0.186.254:42821)
0806 22:40:19.686771 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.686773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.686944 (+ 171us) server_negotiation.cc:407] Connection header received
0806 22:40:19.687453 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.687454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.687464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.687525 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.688354 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.689031 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.690076 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.690280 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.690334 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.692296 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.692297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.692298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:19.692305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.693348 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.693752 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.693753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.693754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.693818 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.694117 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.694118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.694118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.694249 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.694317 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.694597 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.694677 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:19.694905 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.686699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42821 (local address 127.0.186.195:46965)
0806 22:40:19.686839 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.686877 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.687378 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.687579 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.687582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.687812 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:19.688262 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.688266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.689114 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.689115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.689996 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.689998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.690346 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.690347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.690474 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.690891 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:19.690900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:19.692209 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:19.693419 (+ 1210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.693425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.693427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.693672 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.693874 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:19.693874 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:19.693875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:19.693997 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:19.694339 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:19.694346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:19.694468 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.694563 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.694688 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260806 22:40:19.695346 7859 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42821
I20260806 22:40:19.695442 7859 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:19.695583 7859 heartbeater.cc:511] Master 127.0.186.254:42821 requested a full tablet report, sending...
I20260806 22:40:19.695964 7140 ts_manager.cc:194] Registered new tserver with Master: f62d295c37d047bd8b38e34c0f1b64e9 (127.0.186.195:46707)
I20260806 22:40:19.696334 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:46965
I20260806 22:40:19.700277 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:19.706710 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.701437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.0.186.254:42821)
0806 22:40:19.701494 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.701496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.701595 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:40:19.701657 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.701658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.701667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.701711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.702220 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.702672 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.703365 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.703562 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.703614 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.704625 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.704626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.704627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:19.704634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.705669 (+ 1035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.705962 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.705963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.705964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.706042 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:19.706205 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:19.706206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:19.706206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:19.706345 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.706430 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.706558 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.706629 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:40:19.708005 7140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:19.709539 7140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:19.715480 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46707 (local address 127.0.0.1:52180)
0806 22:40:19.712074 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.712081 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.712123 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.712262 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.712263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.712265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:19.712340 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.712342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713249 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.713250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.714415 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.714418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715162 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.715252 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715287 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.715357 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.715412 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":1,"thread_start_us":43,"threads_started":1}
I20260806 22:40:19.715494 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.0.186.195:46707)
0806 22:40:19.711833 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.711836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.712137 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:40:19.712179 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.712181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.712195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.712260 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:19.712421 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713163 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.714517 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715087 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715147 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715202 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.715372 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.715410 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260806 22:40:19.715576 7874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.0.186.194:44583)
0806 22:40:19.711747 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.711752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.711937 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:40:19.712075 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.712077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.712094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.712158 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:19.712435 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713527 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.714537 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715141 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715186 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715233 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.715463 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.715500 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260806 22:40:19.715801 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:44583 (local address 127.0.0.1:57072)
0806 22:40:19.711842 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.711882 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.711954 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.712254 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.712257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.712260 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:19.712347 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.712349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713648 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.713649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.714446 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.714448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715314 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.715375 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715379 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.715681 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.715734 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":51,"thread_start_us":67,"threads_started":1}
I20260806 22:40:19.716005 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.0.186.193:33411)
0806 22:40:19.711728 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.711733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.711963 (+ 230us) server_negotiation.cc:407] Connection header received
0806 22:40:19.712041 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.712044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.712060 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.712151 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:19.712429 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713527 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.714797 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715524 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715578 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715629 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.715847 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.715925 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260806 22:40:19.715919 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.711431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:33411 (local address 127.0.0.1:49286)
0806 22:40:19.711842 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.711882 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.711954 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.712254 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.712257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.712263 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:19.712348 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.712349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.713632 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.713633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.714716 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.714719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.715666 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.715667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.715727 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.715729 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.715778 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.715808 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":33,"thread_start_us":59,"threads_started":1}
I20260806 22:40:19.716717 7757 tablet_service.cc:1467] Processing CreateTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb (DEFAULT_TABLE table=test-table [id=d39e73cdb879495cad5bb1e9e65e855a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:19.716679 7540 tablet_service.cc:1467] Processing CreateTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb (DEFAULT_TABLE table=test-table [id=d39e73cdb879495cad5bb1e9e65e855a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:19.716763 7328 tablet_service.cc:1467] Processing CreateTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb (DEFAULT_TABLE table=test-table [id=d39e73cdb879495cad5bb1e9e65e855a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:19.717028 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.717037 7540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.717060 7328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.719450 7880 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Bootstrap starting.
I20260806 22:40:19.720125 7880 tablet_bootstrap.cc:654] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.720217 7881 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: Bootstrap starting.
I20260806 22:40:19.720854 7881 tablet_bootstrap.cc:654] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.720882 7880 log.cc:824] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.721408 7881 log.cc:824] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.721638 7880 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: No bootstrap required, opened a new log
I20260806 22:40:19.721710 7880 ts_tablet_manager.cc:1397] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:19.721771 7882 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Bootstrap starting.
I20260806 22:40:19.722117 7881 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: No bootstrap required, opened a new log
I20260806 22:40:19.722177 7881 ts_tablet_manager.cc:1397] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:19.722393 7882 tablet_bootstrap.cc:654] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.722978 7882 log.cc:824] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.723824 7882 tablet_bootstrap.cc:492] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: No bootstrap required, opened a new log
I20260806 22:40:19.723749 7881 raft_consensus.cc:348] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.723882 7882 ts_tablet_manager.cc:1397] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:19.723809 7880 raft_consensus.cc:348] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.723917 7880 raft_consensus.cc:374] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.723910 7881 raft_consensus.cc:374] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.723937 7880 raft_consensus.cc:729] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62d295c37d047bd8b38e34c0f1b64e9, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.723946 7881 raft_consensus.cc:729] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78083cef8822453d874d14af7fad23ce, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.724031 7880 consensus_queue.cc:260] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.724071 7881 consensus_queue.cc:260] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.724284 7859 heartbeater.cc:503] Master 127.0.186.254:42821 was elected leader, sending a full tablet report...
I20260806 22:40:19.724331 7881 ts_tablet_manager.cc:1428] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:40:19.724457 7426 heartbeater.cc:503] Master 127.0.186.254:42821 was elected leader, sending a full tablet report...
I20260806 22:40:19.724277 7880 ts_tablet_manager.cc:1428] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:19.725682 7882 raft_consensus.cc:348] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.725785 7882 raft_consensus.cc:374] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.725803 7882 raft_consensus.cc:729] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adbc97f35c5e49cc8176aacca603adb1, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.725879 7882 consensus_queue.cc:260] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.726097 7882 ts_tablet_manager.cc:1428] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:19.726143 7649 heartbeater.cc:503] Master 127.0.186.254:42821 was elected leader, sending a full tablet report...
I20260806 22:40:19.777572 7890 raft_consensus.cc:482] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:19.777688 7890 raft_consensus.cc:504] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.778051 7890 leader_election.cc:288] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adbc97f35c5e49cc8176aacca603adb1 (127.0.186.194:44583), 78083cef8822453d874d14af7fad23ce (127.0.186.193:33411)
I20260806 22:40:19.781775 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.778209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:33411 (local address 127.0.186.195:44489)
0806 22:40:19.778249 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.778262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.778304 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.778477 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.778479 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.778480 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:19.778549 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.778551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.779476 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.779477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.780764 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.780767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.781520 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.781521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.781597 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.781599 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.781646 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.781687 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":20}
I20260806 22:40:19.781798 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.778332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:44489 (local address 127.0.186.193:33411)
0806 22:40:19.778370 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.778372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.778379 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:19.778403 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.778404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.778411 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.778452 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:19.778621 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.779401 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.780846 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.781447 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.781502 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.781509 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.781686 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.781726 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:19.781792 7874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.778279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:46697 (local address 127.0.186.194:44583)
0806 22:40:19.778325 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.778327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.778472 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:40:19.778506 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.778507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.778517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.778556 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:19.778692 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.779400 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.780860 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.781428 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.781500 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.781509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.781688 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.781726 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:19.781785 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.778233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:44583 (local address 127.0.186.195:46697)
0806 22:40:19.778394 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.778404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:19.778464 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:19.778568 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:19.778568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:19.778579 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:19.778635 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.778637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.779476 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.779477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.780764 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:19.780767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.781502 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.781503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:19.781581 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.781586 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:19.781675 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:40:19.781702 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":52,"thread_start_us":106,"threads_started":1}
I20260806 22:40:19.782184 7577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b05aa775f5e4ce4a3f66e9ff421fdbb" candidate_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adbc97f35c5e49cc8176aacca603adb1" is_pre_election: true
I20260806 22:40:19.782250 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b05aa775f5e4ce4a3f66e9ff421fdbb" candidate_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78083cef8822453d874d14af7fad23ce" is_pre_election: true
I20260806 22:40:19.782316 7577 raft_consensus.cc:2454] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62d295c37d047bd8b38e34c0f1b64e9 in term 0.
I20260806 22:40:19.782414 7358 raft_consensus.cc:2454] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62d295c37d047bd8b38e34c0f1b64e9 in term 0.
I20260806 22:40:19.782527 7682 leader_election.cc:302] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adbc97f35c5e49cc8176aacca603adb1, f62d295c37d047bd8b38e34c0f1b64e9; no voters:
I20260806 22:40:19.782691 7890 raft_consensus.cc:2781] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:19.782732 7890 raft_consensus.cc:482] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:19.782747 7890 raft_consensus.cc:3037] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.783403 7890 raft_consensus.cc:504] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.783512 7890 leader_election.cc:288] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [CANDIDATE]: Term 1 election: Requested vote from peers adbc97f35c5e49cc8176aacca603adb1 (127.0.186.194:44583), 78083cef8822453d874d14af7fad23ce (127.0.186.193:33411)
I20260806 22:40:19.783670 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b05aa775f5e4ce4a3f66e9ff421fdbb" candidate_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78083cef8822453d874d14af7fad23ce"
I20260806 22:40:19.783679 7577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b05aa775f5e4ce4a3f66e9ff421fdbb" candidate_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adbc97f35c5e49cc8176aacca603adb1"
I20260806 22:40:19.783741 7577 raft_consensus.cc:3037] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.783743 7358 raft_consensus.cc:3037] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.784399 7577 raft_consensus.cc:2454] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62d295c37d047bd8b38e34c0f1b64e9 in term 1.
I20260806 22:40:19.784457 7358 raft_consensus.cc:2454] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62d295c37d047bd8b38e34c0f1b64e9 in term 1.
I20260806 22:40:19.784587 7682 leader_election.cc:302] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adbc97f35c5e49cc8176aacca603adb1, f62d295c37d047bd8b38e34c0f1b64e9; no voters:
I20260806 22:40:19.784667 7890 raft_consensus.cc:2781] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:19.784806 7890 raft_consensus.cc:686] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 1 LEADER]: Becoming Leader. State: Replica: f62d295c37d047bd8b38e34c0f1b64e9, State: Running, Role: LEADER
I20260806 22:40:19.784897 7890 consensus_queue.cc:237] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } }
I20260806 22:40:19.785658 7140 catalog_manager.cc:5495] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f62d295c37d047bd8b38e34c0f1b64e9 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62d295c37d047bd8b38e34c0f1b64e9" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46707 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:19.790944 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.788925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.0.186.195:46707)
0806 22:40:19.788964 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.788965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.788977 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:19.789046 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.789046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.789060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.789113 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:19.789221 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.789646 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.790363 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.790545 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.790600 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.790717 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:19.790803 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:19.790854 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.790867 (+ 13us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.790883 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:19.792532 7358 raft_consensus.cc:1261] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 1 FOLLOWER]: Refusing update from remote peer f62d295c37d047bd8b38e34c0f1b64e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:19.792528 7577 raft_consensus.cc:1261] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 1 FOLLOWER]: Refusing update from remote peer f62d295c37d047bd8b38e34c0f1b64e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 22:40:19.792593 7651 tablet.cc:2369] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:19.792842 7890 consensus_queue.cc:1035] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78083cef8822453d874d14af7fad23ce" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:19.792977 7894 consensus_queue.cc:1035] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adbc97f35c5e49cc8176aacca603adb1" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:19.795020 7897 mvcc.cc:205] Tried to move back new op lower bound from 7315685457067941888 to 7315685457039597568. Current Snapshot: MvccSnapshot[applied={T|T < 7315685457067941888}]
I20260806 22:40:19.797542 7137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:19.797641 7137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:19.798336 7137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4697edd1fdd84163a6633edd267d6a57: Sending DeleteTablet for 3 replicas of tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb
I20260806 22:40:19.798664 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:19 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:52180
I20260806 22:40:19.798667 7328 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:19 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49286
I20260806 22:40:19.798890 7540 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:19 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:57072
I20260806 22:40:19.798972 7912 tablet_replica.cc:330] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce: stopping tablet replica
I20260806 22:40:19.799064 7912 raft_consensus.cc:2229] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P 78083cef8822453d874d14af7fad23ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:19.799160 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 7234
I20260806 22:40:19.805567 7914 tablet_replica.cc:330] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: stopping tablet replica
I20260806 22:40:19.805644 7911 tablet_replica.cc:330] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: stopping tablet replica
I20260806 22:40:19.805676 7914 raft_consensus.cc:2229] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:19.805712 7911 raft_consensus.cc:2229] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:19.805840 7911 raft_consensus.cc:2258] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:19.805850 7914 raft_consensus.cc:2258] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:19.808411 7911 ts_tablet_manager.cc:1905] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:19.809902 7911 ts_tablet_manager.cc:1918] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:19.809971 7911 log.cc:1196] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/6b05aa775f5e4ce4a3f66e9ff421fdbb
I20260806 22:40:19.810240 7911 ts_tablet_manager.cc:1939] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P f62d295c37d047bd8b38e34c0f1b64e9: Deleting consensus metadata
I20260806 22:40:19.810616 7115 catalog_manager.cc:4841] TS f62d295c37d047bd8b38e34c0f1b64e9 (127.0.186.195:46707): tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb (table test-table [id=d39e73cdb879495cad5bb1e9e65e855a]) successfully deleted
I20260806 22:40:19.812256 7914 ts_tablet_manager.cc:1905] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:19.813704 7914 ts_tablet_manager.cc:1918] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:19.813779 7914 log.cc:1196] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/6b05aa775f5e4ce4a3f66e9ff421fdbb
I20260806 22:40:19.814015 7914 ts_tablet_manager.cc:1939] T 6b05aa775f5e4ce4a3f66e9ff421fdbb P adbc97f35c5e49cc8176aacca603adb1: Deleting consensus metadata
I20260806 22:40:19.814466 7119 catalog_manager.cc:4841] TS adbc97f35c5e49cc8176aacca603adb1 (127.0.186.194:44583): tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb (table test-table [id=d39e73cdb879495cad5bb1e9e65e855a]) successfully deleted
W20260806 22:40:19.816581 7115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:33411 (error 108)
I20260806 22:40:19.816792 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 7434
I20260806 22:40:19.818501 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.816992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:33411 (local address 127.0.0.1:49290)
0806 22:40:19.818414 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.818467 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:33411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1398}
W20260806 22:40:19.824451 7115 catalog_manager.cc:4584] TS 78083cef8822453d874d14af7fad23ce (127.0.186.193:33411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b05aa775f5e4ce4a3f66e9ff421fdbb: Network error: Client connection negotiation failed: client connection to 127.0.186.193:33411: connect: Connection refused (error 111)
I20260806 22:40:19.828385 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 7656
I20260806 22:40:19.834334 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 7088
2026-08-06T22:40:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2025 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:19Z Disabled control of system clock
I20260806 22:40:19.866732 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:39151
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:39151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:40:19.946292 7922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.946477 7922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.946497 7922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.947633 7922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:19.947671 7922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.947682 7922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.947705 7922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:19.947729 7922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:19.949265 7922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:39151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:19.949513 7922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.949795 7922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:19.954407 7922 server_base.cc:1034] running on GCE node
W20260806 22:40:19.954427 7935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.954385 7937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.954380 7934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.954872 7922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.955144 7922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.956282 7922 hybrid_clock.cc:648] HybridClock initialized: now 1786056019956272 us; error 26 us; skew 500 ppm
I20260806 22:40:19.957365 7922 webserver.cc:460] Webserver started at http://127.0.186.254:38253/ using document root <none> and password file <none>
I20260806 22:40:19.957595 7922 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.957645 7922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.957774 7922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.959012 7922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "5450d899dbc3485f89afac82bc441a31"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "d0e555ce1cd1428481a0a532c4a30d1c"
server_key_iv: "be2376e2acee68b1dc971b16052f7deb"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.959373 7922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "5450d899dbc3485f89afac82bc441a31"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "d0e555ce1cd1428481a0a532c4a30d1c"
server_key_iv: "be2376e2acee68b1dc971b16052f7deb"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.960628 7922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:19.961344 7945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.961499 7922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:19.961581 7922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "5450d899dbc3485f89afac82bc441a31"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-gjrz"
server_key: "d0e555ce1cd1428481a0a532c4a30d1c"
server_key_iv: "be2376e2acee68b1dc971b16052f7deb"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.961648 7922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.964936 7922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.965185 7922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.965302 7922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.968979 7922 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:39151
I20260806 22:40:19.969004 8033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:39151 every 8 connection(s)
I20260806 22:40:19.969310 7922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:19.969913 8035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.970655 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 7922
I20260806 22:40:19.970760 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:19.970927 747 external_mini_cluster.cc:1428] Setting key facf7fe436fb68aeab8a8f18ee892736
I20260806 22:40:19.973153 8035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: Bootstrap starting.
I20260806 22:40:19.973379 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.971453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.0.186.254:39151)
0806 22:40:19.971659 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.971668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.971701 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:40:19.972205 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.972214 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.972265 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.972411 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.972980 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.972984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.972984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:19.972989 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.973033 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:19.973046 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.973129 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.973162 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.973215 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260806 22:40:19.973881 8035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.974138 8035 log.cc:824] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.974740 8035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: No bootstrap required, opened a new log
I20260806 22:40:19.976104 8035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } }
I20260806 22:40:19.976231 8035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.976251 8035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5450d899dbc3485f89afac82bc441a31, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.976351 8035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } }
I20260806 22:40:19.976428 8035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:19.976449 8035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:19.976480 8035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.976926 8035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } }
I20260806 22:40:19.977008 8035 leader_election.cc:302] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5450d899dbc3485f89afac82bc441a31; no voters:
I20260806 22:40:19.977124 8035 leader_election.cc:288] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:19.977341 8035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:19.977398 8041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:19.977550 8041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [term 1 LEADER]: Becoming Leader. State: Replica: 5450d899dbc3485f89afac82bc441a31, State: Running, Role: LEADER
I20260806 22:40:19.977691 8041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } }
I20260806 22:40:19.978128 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5450d899dbc3485f89afac82bc441a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } } }
I20260806 22:40:19.978216 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.978387 8043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5450d899dbc3485f89afac82bc441a31. Latest consensus state: current_term: 1 leader_uuid: "5450d899dbc3485f89afac82bc441a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5450d899dbc3485f89afac82bc441a31" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 39151 } } }
I20260806 22:40:19.978421 8059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:19.978454 8043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.978933 8059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:19.980326 8059 catalog_manager.cc:1347] Generated new cluster ID: aa4018cf3a784a6db8fabda13a073dfc
I20260806 22:40:19.980374 8059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:20.002488 8059 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:20.002648 8059 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:20.005185 8059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: Generated new TSK 0
I20260806 22:40:20.005332 8059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:20.012526 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--builtin_ntp_servers=127.0.186.212:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:40:20.094326 8072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:20.094513 8072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:20.094535 8072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:20.095718 8072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:20.095770 8072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:20.095798 8072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:20.097154 8072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:20.097396 8072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:20.097678 8072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:20.102403 8078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.102414 8084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.102535 8081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:20.102766 8072 server_base.cc:1034] running on GCE node
I20260806 22:40:20.103029 8072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:20.103302 8072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:20.104466 8072 hybrid_clock.cc:648] HybridClock initialized: now 1786056020104446 us; error 33 us; skew 500 ppm
I20260806 22:40:20.105695 8072 webserver.cc:460] Webserver started at http://127.0.186.193:34227/ using document root <none> and password file <none>
I20260806 22:40:20.105929 8072 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:20.106007 8072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:20.106148 8072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:20.107218 8072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "9817d28aa0a8483ab655c415872a7f64"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "94e052e221a5b4b0a2d3d8775b2fbf30"
server_key_iv: "7d3280ee3841d3d27dbe885f451e5f57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.107625 8072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "9817d28aa0a8483ab655c415872a7f64"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "94e052e221a5b4b0a2d3d8775b2fbf30"
server_key_iv: "7d3280ee3841d3d27dbe885f451e5f57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.108867 8072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:20.109586 8094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.109789 8072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:20.109874 8072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "9817d28aa0a8483ab655c415872a7f64"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "94e052e221a5b4b0a2d3d8775b2fbf30"
server_key_iv: "7d3280ee3841d3d27dbe885f451e5f57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.109946 8072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:20.113273 8072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:20.113514 8072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:20.113629 8072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:20.113812 8072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:20.114104 8072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:20.114137 8072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.114163 8072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:20.114189 8072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.119411 8072 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:35275
I20260806 22:40:20.119451 8278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:35275 every 8 connection(s)
I20260806 22:40:20.119742 8072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:20.122463 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.120435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:45189 (local address 127.0.186.254:39151)
0806 22:40:20.120479 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.120481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.120686 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:40:20.121122 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.121123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.121129 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.121181 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.122262 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.122264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.122264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.122268 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.122283 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.122287 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.122346 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.122401 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.122413 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:20.122624 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.120400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:39151 (local address 127.0.186.193:45189)
0806 22:40:20.120567 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.120606 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.121033 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.121234 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.121237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.121245 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.122109 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.122109 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.122136 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.122165 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.122348 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.122350 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.122352 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.122398 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.122443 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260806 22:40:20.123121 8281 heartbeater.cc:347] Connected to a master server at 127.0.186.254:39151
I20260806 22:40:20.123194 8281 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:20.123332 8281 heartbeater.cc:511] Master 127.0.186.254:39151 requested a full tablet report, sending...
I20260806 22:40:20.123785 7978 ts_manager.cc:194] Registered new tserver with Master: 9817d28aa0a8483ab655c415872a7f64 (127.0.186.193:35275)
I20260806 22:40:20.127760 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 8072
I20260806 22:40:20.127840 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:20.127993 747 external_mini_cluster.cc:1428] Setting key beca78c80b8f9e9a88f9f25d7105951a
I20260806 22:40:20.128808 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--builtin_ntp_servers=127.0.186.212:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:40:20.209363 8284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:20.209553 8284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:20.209573 8284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:20.210690 8284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:20.210726 8284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:20.210753 8284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:20.212091 8284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:20.212327 8284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:20.212607 8284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:20.217151 8295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.217165 8292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.217262 8290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:20.217639 8284 server_base.cc:1034] running on GCE node
I20260806 22:40:20.217837 8284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:20.218099 8284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:20.219254 8284 hybrid_clock.cc:648] HybridClock initialized: now 1786056020219237 us; error 28 us; skew 500 ppm
I20260806 22:40:20.220541 8284 webserver.cc:460] Webserver started at http://127.0.186.194:36011/ using document root <none> and password file <none>
I20260806 22:40:20.220765 8284 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:20.220814 8284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:20.220949 8284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:20.221989 8284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "13fcad4e90c740d4820129e2bb201e66"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "a6578547c3d8f33d993f0210ea9b12ea"
server_key_iv: "8abf6a41cec56f810aed56ff74cf8799"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.222332 8284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "13fcad4e90c740d4820129e2bb201e66"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "a6578547c3d8f33d993f0210ea9b12ea"
server_key_iv: "8abf6a41cec56f810aed56ff74cf8799"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.223546 8284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:20.224269 8303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.224428 8284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:20.224521 8284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "13fcad4e90c740d4820129e2bb201e66"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "a6578547c3d8f33d993f0210ea9b12ea"
server_key_iv: "8abf6a41cec56f810aed56ff74cf8799"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.224584 8284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:20.227779 8284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:20.228029 8284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:20.228143 8284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:20.228331 8284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:20.228609 8284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:20.228642 8284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.228667 8284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:20.228693 8284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.234050 8284 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:42267
I20260806 22:40:20.234110 8495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:42267 every 8 connection(s)
I20260806 22:40:20.234409 8284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:20.237063 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.235122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:56311 (local address 127.0.186.254:39151)
0806 22:40:20.235178 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.235179 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.235403 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:40:20.235817 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.235818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.235823 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.235870 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.236890 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.236891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.236891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.236895 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.236910 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.236913 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.236954 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.237001 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.237012 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:20.237238 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.235077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:39151 (local address 127.0.186.194:56311)
0806 22:40:20.235300 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.235337 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.235742 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.235943 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.235947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.235954 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.236752 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.236753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.236779 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.236806 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.236954 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.236957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.236960 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.237008 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.237055 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260806 22:40:20.237700 8496 heartbeater.cc:347] Connected to a master server at 127.0.186.254:39151
I20260806 22:40:20.237782 8496 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:20.237921 8496 heartbeater.cc:511] Master 127.0.186.254:39151 requested a full tablet report, sending...
I20260806 22:40:20.238260 7978 ts_manager.cc:194] Registered new tserver with Master: 13fcad4e90c740d4820129e2bb201e66 (127.0.186.194:42267)
I20260806 22:40:20.243664 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 8284
I20260806 22:40:20.243753 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:20.243930 747 external_mini_cluster.cc:1428] Setting key 8c7daf6de9f2d917b315283ac0b138c0
I20260806 22:40:20.244827 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--builtin_ntp_servers=127.0.186.212:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:40:20.324680 8500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:20.324877 8500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:20.324896 8500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:20.326018 8500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:20.326061 8500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:20.326085 8500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:20.327466 8500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:20.327685 8500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:20.327970 8500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:20.332765 8510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:20.332828 8500 server_base.cc:1034] running on GCE node
W20260806 22:40:20.332772 8514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.332772 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:20.333242 8500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:20.333449 8500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:20.334604 8500 hybrid_clock.cc:648] HybridClock initialized: now 1786056020334583 us; error 29 us; skew 500 ppm
I20260806 22:40:20.335860 8500 webserver.cc:460] Webserver started at http://127.0.186.195:42133/ using document root <none> and password file <none>
I20260806 22:40:20.336102 8500 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:20.336151 8500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:20.336277 8500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:20.337316 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "61cc7973c66843b68c52f5ee372d2b8d"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "8f3975c3d77cad74101a8c2f20618879"
server_key_iv: "9998c081d4f6358525a6b59c4a5aef57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.337641 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "61cc7973c66843b68c52f5ee372d2b8d"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "8f3975c3d77cad74101a8c2f20618879"
server_key_iv: "9998c081d4f6358525a6b59c4a5aef57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.338845 8500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:20.339623 8524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.339821 8500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:20.339900 8500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "61cc7973c66843b68c52f5ee372d2b8d"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "8f3975c3d77cad74101a8c2f20618879"
server_key_iv: "9998c081d4f6358525a6b59c4a5aef57"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.339982 8500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:20.343240 8500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:20.343497 8500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:20.343592 8500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:20.343739 8500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:20.343986 8500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:20.344014 8500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.344038 8500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:20.344054 8500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.349318 8500 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:37787
I20260806 22:40:20.349371 8709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:37787 every 8 connection(s)
I20260806 22:40:20.349646 8500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:20.350016 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 8500
I20260806 22:40:20.350118 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:20.350265 747 external_mini_cluster.cc:1428] Setting key a5135fe9fd56875e3a30a6050a4ba253
I20260806 22:40:20.352504 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.350377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:43221 (local address 127.0.186.254:39151)
0806 22:40:20.350452 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.350454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.350679 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:40:20.351129 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.351130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.351136 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.351177 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.352282 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.352283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.352283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.352288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.352303 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.352307 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.352355 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.352425 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.352439 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:40:20.352689 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.350374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:39151 (local address 127.0.186.195:43221)
0806 22:40:20.350538 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.350577 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.351025 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.351243 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.351247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.351256 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.352143 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.352143 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.352170 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.352197 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.352360 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.352364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.352367 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.352435 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.352495 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260806 22:40:20.353073 8711 heartbeater.cc:347] Connected to a master server at 127.0.186.254:39151
I20260806 22:40:20.353153 8711 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:20.353263 8711 heartbeater.cc:511] Master 127.0.186.254:39151 requested a full tablet report, sending...
I20260806 22:40:20.353567 7978 ts_manager.cc:194] Registered new tserver with Master: 61cc7973c66843b68c52f5ee372d2b8d (127.0.186.195:37787)
I20260806 22:40:20.361837 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:20.364048 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.363149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.0.186.254:39151)
0806 22:40:20.363195 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.363197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.363295 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:40:20.363374 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.363375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.363379 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.363426 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.363899 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.363900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.363901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.363904 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.363916 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.363920 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.363959 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.363990 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.364001 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:20.365425 7978 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:20.366912 7978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:20.370530 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.368865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42267 (local address 127.0.0.1:39452)
0806 22:40:20.369291 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.369300 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.369357 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.369500 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.369500 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.369503 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.370129 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.370129 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.370137 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.370149 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.370376 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.370384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.370387 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.370456 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.370469 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":79,"threads_started":1}
I20260806 22:40:20.370553 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.368981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.0.186.194:42267)
0806 22:40:20.369213 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.369218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.369362 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:40:20.369402 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.369404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.369414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.369485 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.370229 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.370237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.370237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.370242 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.370280 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.370312 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.370374 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.370461 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.370477 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260806 22:40:20.370857 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.369009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35275 (local address 127.0.0.1:52968)
0806 22:40:20.369700 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.369709 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.369754 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.370153 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.370154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.370156 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.370494 (+ 338us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.370495 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.370500 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.370510 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.370726 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.370727 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.370728 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.370786 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.370799 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":71,"threads_started":1}
I20260806 22:40:20.370932 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.368831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:37787 (local address 127.0.0.1:56410)
0806 22:40:20.369014 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.369056 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.369109 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.369439 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.369446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.369452 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.369974 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.369975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.369991 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.370012 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.370777 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.370778 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.370779 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.370825 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.370838 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":49,"threads_started":1}
I20260806 22:40:20.371145 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.369122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.0.186.193:35275)
0806 22:40:20.369293 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.369297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.369810 (+ 513us) server_negotiation.cc:407] Connection header received
0806 22:40:20.369838 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.369841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.369852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.369913 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.370592 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.370600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.370600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.370605 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.370636 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.370658 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.370715 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.371043 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.371061 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260806 22:40:20.371253 8383 tablet_service.cc:1467] Processing CreateTablet for tablet 2aefb0a121e24fcb94416608608a0974 (DEFAULT_TABLE table=test-table [id=7e7d7f688720492cb343bfcfd900711f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:20.371624 8383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aefb0a121e24fcb94416608608a0974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:20.371810 8173 tablet_service.cc:1467] Processing CreateTablet for tablet 2aefb0a121e24fcb94416608608a0974 (DEFAULT_TABLE table=test-table [id=7e7d7f688720492cb343bfcfd900711f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:20.372133 8173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aefb0a121e24fcb94416608608a0974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:20.373941 8730 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Bootstrap starting.
I20260806 22:40:20.374183 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.369138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.0.186.195:37787)
0806 22:40:20.369293 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.369297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.369308 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:20.369334 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.369336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.369345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.369419 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.370359 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.370362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.370362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.370365 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.370398 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.370411 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.370766 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.370830 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.370844 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260806 22:40:20.374554 8731 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Bootstrap starting.
I20260806 22:40:20.374621 8730 tablet_bootstrap.cc:654] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:20.374770 8602 tablet_service.cc:1467] Processing CreateTablet for tablet 2aefb0a121e24fcb94416608608a0974 (DEFAULT_TABLE table=test-table [id=7e7d7f688720492cb343bfcfd900711f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:20.374925 8730 log.cc:824] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:20.375006 8602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aefb0a121e24fcb94416608608a0974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:20.375200 8731 tablet_bootstrap.cc:654] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:20.375491 8731 log.cc:824] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:20.375756 8730 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: No bootstrap required, opened a new log
I20260806 22:40:20.375834 8730 ts_tablet_manager.cc:1397] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:20.376137 8731 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: No bootstrap required, opened a new log
I20260806 22:40:20.376196 8731 ts_tablet_manager.cc:1397] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:20.378304 8730 raft_consensus.cc:348] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.378391 8736 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Bootstrap starting.
I20260806 22:40:20.378474 8730 raft_consensus.cc:374] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:20.378520 8730 raft_consensus.cc:729] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcad4e90c740d4820129e2bb201e66, State: Initialized, Role: FOLLOWER
I20260806 22:40:20.378188 8731 raft_consensus.cc:348] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.378575 8731 raft_consensus.cc:374] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:20.378643 8731 raft_consensus.cc:729] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9817d28aa0a8483ab655c415872a7f64, State: Initialized, Role: FOLLOWER
I20260806 22:40:20.378651 8730 consensus_queue.cc:260] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.378780 8731 consensus_queue.cc:260] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.378907 8730 ts_tablet_manager.cc:1428] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:20.378968 8496 heartbeater.cc:503] Master 127.0.186.254:39151 was elected leader, sending a full tablet report...
I20260806 22:40:20.379042 8731 ts_tablet_manager.cc:1428] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:40:20.379088 8281 heartbeater.cc:503] Master 127.0.186.254:39151 was elected leader, sending a full tablet report...
I20260806 22:40:20.379218 8736 tablet_bootstrap.cc:654] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:20.379563 8736 log.cc:824] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:20.380390 8736 tablet_bootstrap.cc:492] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: No bootstrap required, opened a new log
I20260806 22:40:20.380443 8736 ts_tablet_manager.cc:1397] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:20.382249 8736 raft_consensus.cc:348] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.382365 8736 raft_consensus.cc:374] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:20.382383 8736 raft_consensus.cc:729] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61cc7973c66843b68c52f5ee372d2b8d, State: Initialized, Role: FOLLOWER
I20260806 22:40:20.382483 8736 consensus_queue.cc:260] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.382706 8736 ts_tablet_manager.cc:1428] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:40:20.382755 8711 heartbeater.cc:503] Master 127.0.186.254:39151 was elected leader, sending a full tablet report...
W20260806 22:40:20.484907 8497 tablet.cc:2369] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:20.600283 8712 tablet.cc:2369] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:20.619472 8738 raft_consensus.cc:482] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:20.619596 8738 raft_consensus.cc:504] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.619900 8738 leader_election.cc:288] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61cc7973c66843b68c52f5ee372d2b8d (127.0.186.195:37787), 9817d28aa0a8483ab655c415872a7f64 (127.0.186.193:35275)
W20260806 22:40:20.620393 8282 tablet.cc:2369] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:20.621119 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.620064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:37787 (local address 127.0.186.194:40505)
0806 22:40:20.620130 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.620142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.620183 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.620350 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.620350 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.620353 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.620786 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.620787 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.620797 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.620814 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.620990 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.620991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.620992 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.621039 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.621053 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:40:20.621206 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.620142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:40505 (local address 127.0.186.195:37787)
0806 22:40:20.620194 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.620196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.620206 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:20.620238 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.620239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.620245 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.620312 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.620895 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.620896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.620897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.620901 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.620917 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.620922 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.620999 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.621125 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.621140 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:20.621332 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.620243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45723 (local address 127.0.186.193:35275)
0806 22:40:20.620301 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.620304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.620393 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:40:20.620429 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.620430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.620434 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.620486 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.621092 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.621094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.621094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.621099 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.621117 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.621121 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.621168 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.621257 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.621270 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:40:20.621493 8745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.620165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35275 (local address 127.0.186.194:45723)
0806 22:40:20.620339 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.620348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.620389 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.620524 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.620525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.620540 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:20.620968 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:20.620969 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:20.620976 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:20.620989 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:20.621192 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:20.621193 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:20.621193 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 22:40:20.621398 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 22:40:20.621411 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260806 22:40:20.621501 8636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aefb0a121e24fcb94416608608a0974" candidate_uuid: "13fcad4e90c740d4820129e2bb201e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61cc7973c66843b68c52f5ee372d2b8d" is_pre_election: true
I20260806 22:40:20.621656 8636 raft_consensus.cc:2454] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcad4e90c740d4820129e2bb201e66 in term 0.
I20260806 22:40:20.621800 8206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aefb0a121e24fcb94416608608a0974" candidate_uuid: "13fcad4e90c740d4820129e2bb201e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9817d28aa0a8483ab655c415872a7f64" is_pre_election: true
I20260806 22:40:20.621891 8307 leader_election.cc:302] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcad4e90c740d4820129e2bb201e66, 61cc7973c66843b68c52f5ee372d2b8d; no voters:
I20260806 22:40:20.621928 8206 raft_consensus.cc:2454] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcad4e90c740d4820129e2bb201e66 in term 0.
I20260806 22:40:20.622013 8738 raft_consensus.cc:2781] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:20.622072 8738 raft_consensus.cc:482] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:20.622094 8738 raft_consensus.cc:3037] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:20.622774 8738 raft_consensus.cc:504] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.622895 8738 leader_election.cc:288] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [CANDIDATE]: Term 1 election: Requested vote from peers 61cc7973c66843b68c52f5ee372d2b8d (127.0.186.195:37787), 9817d28aa0a8483ab655c415872a7f64 (127.0.186.193:35275)
I20260806 22:40:20.623061 8636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aefb0a121e24fcb94416608608a0974" candidate_uuid: "13fcad4e90c740d4820129e2bb201e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61cc7973c66843b68c52f5ee372d2b8d"
I20260806 22:40:20.623070 8206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aefb0a121e24fcb94416608608a0974" candidate_uuid: "13fcad4e90c740d4820129e2bb201e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9817d28aa0a8483ab655c415872a7f64"
I20260806 22:40:20.623134 8206 raft_consensus.cc:3037] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:20.623134 8636 raft_consensus.cc:3037] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:20.623834 8636 raft_consensus.cc:2454] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcad4e90c740d4820129e2bb201e66 in term 1.
I20260806 22:40:20.623871 8206 raft_consensus.cc:2454] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcad4e90c740d4820129e2bb201e66 in term 1.
I20260806 22:40:20.623992 8307 leader_election.cc:302] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcad4e90c740d4820129e2bb201e66, 61cc7973c66843b68c52f5ee372d2b8d; no voters:
I20260806 22:40:20.624085 8738 raft_consensus.cc:2781] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:20.624229 8738 raft_consensus.cc:686] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcad4e90c740d4820129e2bb201e66, State: Running, Role: LEADER
I20260806 22:40:20.624308 8738 consensus_queue.cc:237] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } }
I20260806 22:40:20.625100 7978 catalog_manager.cc:5495] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcad4e90c740d4820129e2bb201e66 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "13fcad4e90c740d4820129e2bb201e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcad4e90c740d4820129e2bb201e66" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42267 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:20.652951 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.652083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.0.186.194:42267)
0806 22:40:20.652141 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.652143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.652153 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:20.652212 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.652213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.652225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.652270 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.652744 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.652745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.652745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.652750 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.652764 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.652769 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.652818 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.652866 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.652878 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:40:20.654595 8636 raft_consensus.cc:1261] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 1 FOLLOWER]: Refusing update from remote peer 13fcad4e90c740d4820129e2bb201e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:20.654595 8206 raft_consensus.cc:1261] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 1 FOLLOWER]: Refusing update from remote peer 13fcad4e90c740d4820129e2bb201e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:20.654892 8738 consensus_queue.cc:1035] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61cc7973c66843b68c52f5ee372d2b8d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:20.655041 8746 consensus_queue.cc:1035] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9817d28aa0a8483ab655c415872a7f64" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:20.657080 8755 mvcc.cc:205] Tried to move back new op lower bound from 7315685460598767616 to 7315685460477661184. Current Snapshot: MvccSnapshot[applied={T|T < 7315685460598767616}]
I20260806 22:40:20.657327 8757 mvcc.cc:205] Tried to move back new op lower bound from 7315685460598767616 to 7315685460477661184. Current Snapshot: MvccSnapshot[applied={T|T < 7315685460598767616}]
I20260806 22:40:20.657866 8756 mvcc.cc:205] Tried to move back new op lower bound from 7315685460598767616 to 7315685460477661184. Current Snapshot: MvccSnapshot[applied={T|T < 7315685460598767616 or (T in {7315685460598767616})}]
I20260806 22:40:20.659480 7978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:20.659576 7978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:20.660261 7978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5450d899dbc3485f89afac82bc441a31: Sending DeleteTablet for 3 replicas of tablet 2aefb0a121e24fcb94416608608a0974
I20260806 22:40:20.660553 8602 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aefb0a121e24fcb94416608608a0974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:20 UTC) from {username='slave'} at 127.0.0.1:56410
I20260806 22:40:20.660632 8173 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aefb0a121e24fcb94416608608a0974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:20 UTC) from {username='slave'} at 127.0.0.1:52968
I20260806 22:40:20.660765 8383 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aefb0a121e24fcb94416608608a0974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:20 UTC) from {username='slave'} at 127.0.0.1:39452
I20260806 22:40:20.660842 8770 tablet_replica.cc:330] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: stopping tablet replica
I20260806 22:40:20.660807 8768 tablet_replica.cc:330] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: stopping tablet replica
I20260806 22:40:20.660948 8770 raft_consensus.cc:2229] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:20.661046 8768 raft_consensus.cc:2229] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:20.661072 8770 raft_consensus.cc:2258] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:20.661271 8768 raft_consensus.cc:2258] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:20.661262 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 8072
I20260806 22:40:20.661495 8770 ts_tablet_manager.cc:1905] T 2aefb0a121e24fcb94416608608a0974 P 9817d28aa0a8483ab655c415872a7f64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:20.663897 8768 ts_tablet_manager.cc:1905] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:20.665489 8768 ts_tablet_manager.cc:1918] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:20.665588 8768 log.cc:1196] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/2aefb0a121e24fcb94416608608a0974
I20260806 22:40:20.665871 8768 ts_tablet_manager.cc:1939] T 2aefb0a121e24fcb94416608608a0974 P 61cc7973c66843b68c52f5ee372d2b8d: Deleting consensus metadata
I20260806 22:40:20.666792 7950 catalog_manager.cc:4841] TS 61cc7973c66843b68c52f5ee372d2b8d (127.0.186.195:37787): tablet 2aefb0a121e24fcb94416608608a0974 (table test-table [id=7e7d7f688720492cb343bfcfd900711f]) successfully deleted
I20260806 22:40:20.671406 8771 tablet_replica.cc:330] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66: stopping tablet replica
I20260806 22:40:20.671516 8771 raft_consensus.cc:2229] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:20.671648 8771 raft_consensus.cc:2258] T 2aefb0a121e24fcb94416608608a0974 P 13fcad4e90c740d4820129e2bb201e66 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:40:20.671875 7950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:35275 (error 108)
I20260806 22:40:20.671985 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 8284
I20260806 22:40:20.679858 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.679718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35275 (local address 127.0.0.1:52984)
0806 22:40:20.679772 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.679820 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:35275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:40:20.679986 7950 catalog_manager.cc:4584] TS 9817d28aa0a8483ab655c415872a7f64 (127.0.186.193:35275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aefb0a121e24fcb94416608608a0974: Network error: Client connection negotiation failed: client connection to 127.0.186.193:35275: connect: Connection refused (error 111)
I20260806 22:40:20.685600 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 8500
I20260806 22:40:20.685618 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.685512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42267 (local address 127.0.0.1:39466)
0806 22:40:20.685560 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.685587 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.194:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:40:20.693068 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 7922
2026-08-06T22:40:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (856 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:40:20.716883 747 mini_oidc.cc:152] Starting JWKS server
I20260806 22:40:20.718042 747 webserver.cc:460] Webserver started at https://127.0.0.1:32961/ using document root <none> and password file <none>
I20260806 22:40:20.718133 747 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:40:20.718292 747 webserver.cc:460] Webserver started at http://127.0.0.1:44209/ using document root <none> and password file <none>
2026-08-06T22:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:20Z Disabled control of system clock
I20260806 22:40:20.725946 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37313
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37313
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32961/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/testchainca.cert with env {}
W20260806 22:40:20.805150 8780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:20.805351 8780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:20.805372 8780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:20.806485 8780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:20.806521 8780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:20.806532 8780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:20.806543 8780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:20.806559 8780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:40:20.806576 8780 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:40:20.806586 8780 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32961/jwks.json
I20260806 22:40:20.808087 8780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37313
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32961/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:20.808274 8780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:20.808431 8780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:20.812939 8792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.812955 8791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:20.812951 8794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:20.813036 8780 server_base.cc:1034] running on GCE node
I20260806 22:40:20.813364 8780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:20.813637 8780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:20.814783 8780 hybrid_clock.cc:648] HybridClock initialized: now 1786056020814765 us; error 33 us; skew 500 ppm
I20260806 22:40:20.816030 8780 webserver.cc:460] Webserver started at http://127.0.186.254:45593/ using document root <none> and password file <none>
I20260806 22:40:20.816242 8780 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:20.816293 8780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:20.816423 8780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:20.817404 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "f84b9383c3c446dba8d653e54ef894bf"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "4e15c8fead863688457903b6d99e7fcf"
server_key_iv: "8eeeeda1cf3994e59b1bd350e78e4d70"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.817746 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "f84b9383c3c446dba8d653e54ef894bf"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "4e15c8fead863688457903b6d99e7fcf"
server_key_iv: "8eeeeda1cf3994e59b1bd350e78e4d70"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.818931 8780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:20.819666 8811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.819821 8780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:20.819945 8780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "f84b9383c3c446dba8d653e54ef894bf"
format_stamp: "Formatted at 2026-08-06 22:40:20 on dist-test-slave-gjrz"
server_key: "4e15c8fead863688457903b6d99e7fcf"
server_key_iv: "8eeeeda1cf3994e59b1bd350e78e4d70"
server_key_version: "encryptionkey@0"
I20260806 22:40:20.820011 8780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:20.830265 8780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:20.830507 8780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:20.830626 8780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:20.834308 8780 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37313
I20260806 22:40:20.834338 8898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37313 every 8 connection(s)
I20260806 22:40:20.834626 8780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:20.835654 8899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:20.837833 8899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: Bootstrap starting.
I20260806 22:40:20.838405 8899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:20.838661 8899 log.cc:824] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:20.839344 8899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: No bootstrap required, opened a new log
I20260806 22:40:20.839964 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 8780
I20260806 22:40:20.840099 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:20.840271 747 external_mini_cluster.cc:1428] Setting key 643fe2d487ac1ca26f53299cf3b455e5
I20260806 22:40:20.841265 8899 raft_consensus.cc:348] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } }
I20260806 22:40:20.841432 8899 raft_consensus.cc:374] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:20.841470 8899 raft_consensus.cc:729] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84b9383c3c446dba8d653e54ef894bf, State: Initialized, Role: FOLLOWER
I20260806 22:40:20.841589 8899 consensus_queue.cc:260] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } }
I20260806 22:40:20.841661 8899 raft_consensus.cc:388] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:20.841689 8899 raft_consensus.cc:482] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:20.841725 8899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:20.842389 8899 raft_consensus.cc:504] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } }
I20260806 22:40:20.842496 8899 leader_election.cc:302] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f84b9383c3c446dba8d653e54ef894bf; no voters:
I20260806 22:40:20.842644 8899 leader_election.cc:288] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:20.842687 8905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:20.842831 8905 raft_consensus.cc:686] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [term 1 LEADER]: Becoming Leader. State: Replica: f84b9383c3c446dba8d653e54ef894bf, State: Running, Role: LEADER
I20260806 22:40:20.842871 8899 sys_catalog.cc:564] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:20.842924 8905 consensus_queue.cc:237] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } }
I20260806 22:40:20.843242 8905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f84b9383c3c446dba8d653e54ef894bf. Latest consensus state: current_term: 1 leader_uuid: "f84b9383c3c446dba8d653e54ef894bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } } }
I20260806 22:40:20.843322 8908 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f84b9383c3c446dba8d653e54ef894bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84b9383c3c446dba8d653e54ef894bf" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37313 } } }
I20260806 22:40:20.843343 8905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:20.843374 8908 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf [sys.catalog]: This master's current role is: LEADER
W20260806 22:40:20.844631 8932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:40:20.844676 8932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:40:20.844719 8916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:20.844867 8916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:20.846380 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.840703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.0.186.254:37313)
0806 22:40:20.840925 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.840934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.840971 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:40:20.841234 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.841243 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.841316 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.841453 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:20.841651 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.843342 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.844447 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.845001 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.845114 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:20.845755 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:20.845759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:20.845759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:20.845906 (+ 147us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:20.845938 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:20.845949 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:20.846001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:20.846082 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:40:20.846160 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260806 22:40:20.846870 8916 catalog_manager.cc:1347] Generated new cluster ID: 178283a68f5444f5af5d5fc55958acad
I20260806 22:40:20.846920 8916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:20.847317 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:20.851408 8916 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:20.851858 8916 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:20.856137 8916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f84b9383c3c446dba8d653e54ef894bf: Generated new TSK 0
I20260806 22:40:20.856277 8916 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:40:20.876925 8950 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:40:20.877084 8950 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:40:20.877208 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:20.862195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.0.186.254:37313)
0806 22:40:20.862232 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:20.862233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:20.862382 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:40:20.862490 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:20.862491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:20.862501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:20.862554 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:40:20.862713 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.863154 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.864290 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.864468 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.864529 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:20.864731 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:40:20.876987 (+ 12256us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:40:20.877145 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49168: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 22:40:20.877259 8904 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49168: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:40:20.877765 8948 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:20.862137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37313 (local address 127.0.0.1:49168)
0806 22:40:20.862320 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:20.862330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:20.862396 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:20.862585 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:20.862586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:20.862588 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:40:20.862656 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:20.862658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.863238 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.863238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:20.864199 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:20.864203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:20.864547 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:20.864548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:20.864618 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:20.864641 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:40:20.877232 (+ 12591us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:40:20.877352 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.186.254:37313: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.186.254:37313: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32961/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:40:20.878208 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 8780
2026-08-06T22:40:20Z chronyd exiting
I20260806 22:40:20.934888 747 test_util.cc:175] -----------------------------------------------
I20260806 22:40:20.934962 747 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055997241161-747-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:40:20.935761 747 mini_oidc.cc:152] Starting JWKS server
I20260806 22:40:20.936415 747 webserver.cc:460] Webserver started at https://127.0.0.1:34547/ using document root <none> and password file <none>
I20260806 22:40:20.936455 747 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:40:20.936586 747 webserver.cc:460] Webserver started at http://127.0.0.1:40441/ using document root <none> and password file <none>
2026-08-06T22:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:20Z Disabled control of system clock
I20260806 22:40:20.944788 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35239
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:35239
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34547/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/testchainca.cert with env {}
W20260806 22:40:21.024260 8962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.024457 8962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.024479 8962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.025662 8962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:21.025703 8962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.025714 8962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.025727 8962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:21.025749 8962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:40:21.025765 8962 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:40:21.025786 8962 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34547/jwks.json
I20260806 22:40:21.027231 8962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:35239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34547/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:21.027484 8962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.027675 8962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.032199 8973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.032251 8962 server_base.cc:1034] running on GCE node
W20260806 22:40:21.032258 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.032203 8968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.032634 8962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.032886 8962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.034032 8962 hybrid_clock.cc:648] HybridClock initialized: now 1786056021034018 us; error 30 us; skew 500 ppm
I20260806 22:40:21.035154 8962 webserver.cc:460] Webserver started at http://127.0.186.254:46731/ using document root <none> and password file <none>
I20260806 22:40:21.035388 8962 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.035439 8962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.035557 8962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.036603 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "3266c7bd9a3542739ab8ef2de3033f83"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "af6e38e0c3bcc68ca5ef555352541b04"
server_key_iv: "361468284c2b90e0a76b7ae2799df283"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.036923 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "3266c7bd9a3542739ab8ef2de3033f83"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "af6e38e0c3bcc68ca5ef555352541b04"
server_key_iv: "361468284c2b90e0a76b7ae2799df283"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.038146 8962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:21.038818 8981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.038988 8962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:21.039072 8962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "3266c7bd9a3542739ab8ef2de3033f83"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "af6e38e0c3bcc68ca5ef555352541b04"
server_key_iv: "361468284c2b90e0a76b7ae2799df283"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.039134 8962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.068888 8962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.069198 8962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.069334 8962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.073144 8962 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:35239
I20260806 22:40:21.073181 9069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:35239 every 8 connection(s)
I20260806 22:40:21.073498 8962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:21.074440 9070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:21.076766 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83: Bootstrap starting.
I20260806 22:40:21.077317 9070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:21.077560 9070 log.cc:824] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:21.078153 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83: No bootstrap required, opened a new log
I20260806 22:40:21.078974 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 8962
I20260806 22:40:21.079068 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:21.079243 747 external_mini_cluster.cc:1428] Setting key 854412cae996eca68fc57f79787e312e
I20260806 22:40:21.079982 9070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } }
I20260806 22:40:21.080159 9070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:21.080193 9070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3266c7bd9a3542739ab8ef2de3033f83, State: Initialized, Role: FOLLOWER
I20260806 22:40:21.080317 9070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } }
I20260806 22:40:21.080399 9070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:21.080428 9070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:21.080483 9070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:21.080992 9070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } }
I20260806 22:40:21.081090 9070 leader_election.cc:302] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3266c7bd9a3542739ab8ef2de3033f83; no voters:
I20260806 22:40:21.081239 9070 leader_election.cc:288] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:21.081290 9076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:21.081400 9076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [term 1 LEADER]: Becoming Leader. State: Replica: 3266c7bd9a3542739ab8ef2de3033f83, State: Running, Role: LEADER
I20260806 22:40:21.081517 9070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:21.081535 9076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } }
I20260806 22:40:21.081921 9078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3266c7bd9a3542739ab8ef2de3033f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } } }
I20260806 22:40:21.082015 9078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.082212 9080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3266c7bd9a3542739ab8ef2de3033f83. Latest consensus state: current_term: 1 leader_uuid: "3266c7bd9a3542739ab8ef2de3033f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3266c7bd9a3542739ab8ef2de3033f83" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35239 } } }
I20260806 22:40:21.082329 9080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.082602 9085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:21.083076 9085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:21.084630 9085 catalog_manager.cc:1347] Generated new cluster ID: c3560699974c4d7ab242dd28faed863a
I20260806 22:40:21.084987 9085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:21.085316 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.079740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.0.186.254:35239)
0806 22:40:21.079969 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.079979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.080016 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:40:21.080127 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.080134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.080203 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.080356 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.080583 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.081986 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.083028 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.083657 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.083774 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.084496 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.084500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.084500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:21.084691 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.084733 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:21.084747 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.084802 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.085031 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.085113 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260806 22:40:21.086225 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:21.091889 9085 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:21.092424 9085 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:21.100741 9085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3266c7bd9a3542739ab8ef2de3033f83: Generated new TSK 0
I20260806 22:40:21.100919 9085 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:40:21.125104 9116 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:40:21.125185 9116 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:40:21.125445 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.111566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.0.186.254:35239)
0806 22:40:21.111612 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.111614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.111696 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:40:21.111766 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.111767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.111777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.111833 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:40:21.111988 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.112470 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.113588 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.113765 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.113806 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.113971 (+ 165us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:40:21.125275 (+ 11304us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34547/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:40:21.125397 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34547/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:40:21.125488 9075 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34547/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:40:21.125613 9114 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:21.111485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35239 (local address 127.0.0.1:40540)
0806 22:40:21.111630 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:21.111641 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:21.111705 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:21.111843 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:21.111844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:21.111846 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:40:21.111901 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.111903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.112540 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.112540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.113503 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.113507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.113823 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.113824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.113888 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.113893 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:40:21.125383 (+ 11490us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34547/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:40:21.125509 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.186.254:35239: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34547/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 22:40:21.125875 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 8962
2026-08-06T22:40:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:40:21.174501 747 mini_oidc.cc:152] Starting JWKS server
I20260806 22:40:21.175154 747 webserver.cc:460] Webserver started at https://127.0.0.1:34295/ using document root <none> and password file <none>
I20260806 22:40:21.175196 747 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:40:21.175354 747 webserver.cc:460] Webserver started at http://127.0.0.1:34041/ using document root <none> and password file <none>
2026-08-06T22:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:21Z Disabled control of system clock
I20260806 22:40:21.183743 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38355
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:38355
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34295/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:40:21.263180 9127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.263409 9127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.263441 9127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.264559 9127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:21.264603 9127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.264621 9127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.264634 9127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:21.264649 9127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:40:21.264674 9127 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:40:21.264698 9127 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34295/jwks.json
I20260806 22:40:21.266005 9127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:38355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34295/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:21.266229 9127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.266471 9127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.270903 9137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.270949 9138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.270923 9140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.271185 9127 server_base.cc:1034] running on GCE node
I20260806 22:40:21.271435 9127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.271723 9127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.272861 9127 hybrid_clock.cc:648] HybridClock initialized: now 1786056021272849 us; error 28 us; skew 500 ppm
I20260806 22:40:21.274066 9127 webserver.cc:460] Webserver started at http://127.0.186.254:39985/ using document root <none> and password file <none>
I20260806 22:40:21.274315 9127 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.274379 9127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.274497 9127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.275662 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "a0e0d2dbab564374a37b2d536b6c1971"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "fef0f274f2726719d680da6645076c60"
server_key_iv: "910ecec0320971150754b88c19d79f20"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.275980 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "a0e0d2dbab564374a37b2d536b6c1971"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "fef0f274f2726719d680da6645076c60"
server_key_iv: "910ecec0320971150754b88c19d79f20"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.277297 9127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:21.277981 9152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.278144 9127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:21.278228 9127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "a0e0d2dbab564374a37b2d536b6c1971"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "fef0f274f2726719d680da6645076c60"
server_key_iv: "910ecec0320971150754b88c19d79f20"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.278290 9127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.298399 9127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.298687 9127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.298810 9127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.302474 9127 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:38355
I20260806 22:40:21.302511 9241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:38355 every 8 connection(s)
I20260806 22:40:21.302809 9127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:21.303506 9243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:21.306195 9243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971: Bootstrap starting.
I20260806 22:40:21.306780 9243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:21.307042 9243 log.cc:824] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:21.307643 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 9127
I20260806 22:40:21.307737 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:21.307744 9243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971: No bootstrap required, opened a new log
I20260806 22:40:21.307893 747 external_mini_cluster.cc:1428] Setting key d4dad85ed8584d33fcaaf04c6f2d464a
I20260806 22:40:21.310460 9243 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } }
I20260806 22:40:21.310626 9243 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:21.310655 9243 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e0d2dbab564374a37b2d536b6c1971, State: Initialized, Role: FOLLOWER
I20260806 22:40:21.310766 9243 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } }
I20260806 22:40:21.310838 9243 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:21.310925 9243 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:21.310979 9243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:21.311553 9243 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } }
I20260806 22:40:21.311654 9243 leader_election.cc:302] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0e0d2dbab564374a37b2d536b6c1971; no voters:
I20260806 22:40:21.311806 9243 leader_election.cc:288] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:21.311851 9249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:21.312001 9249 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [term 1 LEADER]: Becoming Leader. State: Replica: a0e0d2dbab564374a37b2d536b6c1971, State: Running, Role: LEADER
I20260806 22:40:21.312067 9243 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:21.312180 9249 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } }
I20260806 22:40:21.312544 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0e0d2dbab564374a37b2d536b6c1971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } } }
I20260806 22:40:21.312635 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.312824 9250 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0e0d2dbab564374a37b2d536b6c1971. Latest consensus state: current_term: 1 leader_uuid: "a0e0d2dbab564374a37b2d536b6c1971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0e0d2dbab564374a37b2d536b6c1971" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38355 } } }
I20260806 22:40:21.312875 9250 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.312891 9257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:21.313555 9257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:21.314061 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.308365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.0.186.254:38355)
0806 22:40:21.308576 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.308592 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.308626 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:40:21.309041 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.309050 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.309124 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.309269 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.309421 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.310799 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.311685 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.312272 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.312453 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.313151 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.313158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.313158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:21.313384 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.313433 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:21.313450 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.313550 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.313746 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.313832 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 22:40:21.314909 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:21.316258 9257 catalog_manager.cc:1347] Generated new cluster ID: 3c34c996136248bf8734543d948d2f5f
I20260806 22:40:21.316313 9257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:21.340255 9257 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:21.340780 9257 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:21.355966 9257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0e0d2dbab564374a37b2d536b6c1971: Generated new TSK 0
I20260806 22:40:21.356159 9257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:21.370873 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.367555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.0.186.254:38355)
0806 22:40:21.367609 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.367611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.367716 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:40:21.367794 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.367794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.367806 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.367858 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:40:21.367980 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.368472 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.369508 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.369709 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.369763 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.369929 (+ 166us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:40:21.370681 (+ 752us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:40:21.370806 (+ 125us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50724: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 22:40:21.370945 9248 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50724: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:40:21.371017 9288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:21.367496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38355 (local address 127.0.0.1:50724)
0806 22:40:21.367648 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:21.367660 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:21.367728 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:21.367868 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:21.367869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:21.367871 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:40:21.367930 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.367932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.368544 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.368545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.369419 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.369423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.369780 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.369781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.369839 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.369845 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:40:21.370777 (+ 932us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:40:21.370869 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.186.254:38355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260806 22:40:21.371214 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 9127
E20260806 22:40:21.402333 9289 webserver.cc:533] Webserver: sslize failed:
E20260806 22:40:21.402410 9289 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:40:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (255 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:21Z Disabled control of system clock
I20260806 22:40:21.437896 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:41485
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:43619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:41485
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/testchainca.cert with env {}
W20260806 22:40:21.519615 9298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.519806 9298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.519827 9298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.520954 9298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:21.520992 9298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.521003 9298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.521014 9298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:21.521025 9298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:40:21.521037 9298 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:40:21.521062 9298 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:40:21.522538 9298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:41485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:41485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:21.522755 9298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.522936 9298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.527411 9306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.527483 9305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.527493 9310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.527601 9298 server_base.cc:1034] running on GCE node
I20260806 22:40:21.527825 9298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.528092 9298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.529227 9298 hybrid_clock.cc:648] HybridClock initialized: now 1786056021529215 us; error 30 us; skew 500 ppm
I20260806 22:40:21.530400 9298 webserver.cc:460] Webserver started at http://127.0.186.254:42277/ using document root <none> and password file <none>
I20260806 22:40:21.530607 9298 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.530655 9298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.530778 9298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.531822 9298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "ea95aaf43e324c36acb23ae2a00a0e1b"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "5113356663078707ee3f528cf63794d2"
server_key_iv: "83924afb1a950e102905afaa23f41e59"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.532152 9298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "ea95aaf43e324c36acb23ae2a00a0e1b"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "5113356663078707ee3f528cf63794d2"
server_key_iv: "83924afb1a950e102905afaa23f41e59"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.533382 9298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:21.534010 9320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.534163 9298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:21.534245 9298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "ea95aaf43e324c36acb23ae2a00a0e1b"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-gjrz"
server_key: "5113356663078707ee3f528cf63794d2"
server_key_iv: "83924afb1a950e102905afaa23f41e59"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.534307 9298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.566818 9298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.567140 9298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.567318 9298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.571254 9298 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:41485
I20260806 22:40:21.571331 9410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:41485 every 8 connection(s)
I20260806 22:40:21.571645 9298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:21.572193 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 9298
I20260806 22:40:21.572291 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:21.572453 747 external_mini_cluster.cc:1428] Setting key 7b391f4c492dad2dc41578a6dc1dbef8
I20260806 22:40:21.572651 9411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:21.575821 9411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b: Bootstrap starting.
I20260806 22:40:21.576472 9411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:21.576787 9411 log.cc:824] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:21.577521 9411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b: No bootstrap required, opened a new log
I20260806 22:40:21.577881 9416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.572890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.0.186.254:41485)
0806 22:40:21.573096 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.573103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.573128 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:40:21.573742 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.573750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.573809 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.573919 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.574184 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.575022 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.576088 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.576474 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.576582 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.577225 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.577231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.577231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:21.577395 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.577427 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:21.577438 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.577488 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.577610 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.577695 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 22:40:21.578794 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:21.579588 9411 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } }
I20260806 22:40:21.579747 9411 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:21.579797 9411 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea95aaf43e324c36acb23ae2a00a0e1b, State: Initialized, Role: FOLLOWER
I20260806 22:40:21.579912 9411 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } }
I20260806 22:40:21.579986 9411 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:21.580020 9411 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:21.580067 9411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:21.580588 9411 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } }
I20260806 22:40:21.580683 9411 leader_election.cc:302] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea95aaf43e324c36acb23ae2a00a0e1b; no voters:
I20260806 22:40:21.580834 9411 leader_election.cc:288] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:21.580883 9418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:21.581018 9418 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [term 1 LEADER]: Becoming Leader. State: Replica: ea95aaf43e324c36acb23ae2a00a0e1b, State: Running, Role: LEADER
I20260806 22:40:21.581064 9411 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:21.581115 9418 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } }
I20260806 22:40:21.581496 9419 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } } }
I20260806 22:40:21.581516 9422 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea95aaf43e324c36acb23ae2a00a0e1b. Latest consensus state: current_term: 1 leader_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea95aaf43e324c36acb23ae2a00a0e1b" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 41485 } } }
I20260806 22:40:21.581614 9419 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.581621 9422 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.581840 9431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:21.582342 9431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:21.583793 9431 catalog_manager.cc:1347] Generated new cluster ID: be0fd4c192cc4ce1968ceb227a4a61dc
I20260806 22:40:21.583851 9431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:21.595281 9431 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:21.595748 9431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:21.601851 9431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea95aaf43e324c36acb23ae2a00a0e1b: Generated new TSK 0
I20260806 22:40:21.602018 9431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:22.446784 9416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.608875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.0.186.254:41485)
0806 22:40:21.608926 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.608928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.609041 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:40:21.609108 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.609109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.609123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.609180 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:40:21.609305 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.609796 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.610528 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.610719 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.610772 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.610939 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:40:22.446500 (+835561us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:40:22.446699 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42834: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:40:22.446859 9416 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42834: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:40:22.446940 9456 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:21.608807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:41485 (local address 127.0.0.1:42834)
0806 22:40:21.608978 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:21.608987 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:21.609041 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:21.609184 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:21.609185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:21.609186 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:40:21.609246 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.609248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.609864 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.609865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.610450 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.610452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.610790 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.610791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.610850 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.610856 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:40:22.446669 (+835813us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:40:22.446827 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.186.254:41485: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
I20260806 22:40:22.447225 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 9298
2026-08-06T22:40:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1043 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:22 dist-test-slave-gjrz krb5kdc[9475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:22 dist-test-slave-gjrz krb5kdc[9475](info): set up 2 sockets
Aug 06 22:40:22 dist-test-slave-gjrz krb5kdc[9475](info): commencing operation
krb5kdc: starting...
W20260806 22:40:23.798868 747 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:23 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:23.914494 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35233
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:35233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:23.996213 9497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:23.996425 9497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:23.996456 9497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:23.997576 9497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:23.997617 9497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:23.997635 9497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:23.997651 9497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:23.997668 9497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:23.999171 9497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:35233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:23.999425 9497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:23.999625 9497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:24.004166 9507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.004196 9506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.004189 9510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.004438 9497 server_base.cc:1034] running on GCE node
I20260806 22:40:24.004657 9497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:24.004918 9497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:24.006067 9497 hybrid_clock.cc:648] HybridClock initialized: now 1786056024006043 us; error 34 us; skew 500 ppm
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.008563 9497 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:24.009001 9497 webserver.cc:460] Webserver started at http://127.0.186.254:41207/ using document root <none> and password file <none>
I20260806 22:40:24.009173 9497 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:24.009212 9497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:24.009300 9497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:24.010272 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "f2156ee2c16e4f7180d8d33c329925dd"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "e8d77b142b06bb725bda731355d23d86"
server_key_iv: "2b6dc04ef438d6719bfbe292dcc019f3"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.010573 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "f2156ee2c16e4f7180d8d33c329925dd"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "e8d77b142b06bb725bda731355d23d86"
server_key_iv: "2b6dc04ef438d6719bfbe292dcc019f3"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.011847 9497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:24.012495 9520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.012655 9497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:24.012734 9497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "f2156ee2c16e4f7180d8d33c329925dd"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "e8d77b142b06bb725bda731355d23d86"
server_key_iv: "2b6dc04ef438d6719bfbe292dcc019f3"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.012789 9497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:24.033571 9497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:24.035492 9497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:24.035607 9497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:24.039377 9497 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:35233
I20260806 22:40:24.039453 9596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:35233 every 8 connection(s)
I20260806 22:40:24.039929 9497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:24.040630 9598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:24.042974 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd: Bootstrap starting.
I20260806 22:40:24.043596 9598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:24.043872 9598 log.cc:824] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:24.044528 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd: No bootstrap required, opened a new log
I20260806 22:40:24.046134 9598 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } }
I20260806 22:40:24.046276 9598 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:24.046298 9598 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2156ee2c16e4f7180d8d33c329925dd, State: Initialized, Role: FOLLOWER
I20260806 22:40:24.046423 9598 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } }
I20260806 22:40:24.046518 9598 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:24.046550 9598 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:24.046591 9598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:24.047565 9598 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } }
I20260806 22:40:24.047662 9598 leader_election.cc:302] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2156ee2c16e4f7180d8d33c329925dd; no voters:
I20260806 22:40:24.047825 9598 leader_election.cc:288] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:24.047971 9606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:24.048092 9598 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:24.048144 9606 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [term 1 LEADER]: Becoming Leader. State: Replica: f2156ee2c16e4f7180d8d33c329925dd, State: Running, Role: LEADER
I20260806 22:40:24.048269 9606 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } }
I20260806 22:40:24.048645 9607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2156ee2c16e4f7180d8d33c329925dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } } }
I20260806 22:40:24.048661 9609 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2156ee2c16e4f7180d8d33c329925dd. Latest consensus state: current_term: 1 leader_uuid: "f2156ee2c16e4f7180d8d33c329925dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2156ee2c16e4f7180d8d33c329925dd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35233 } } }
I20260806 22:40:24.048744 9607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:24.048748 9609 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:24.049437 9619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:24.049655 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 9497
I20260806 22:40:24.049772 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:24.049988 747 external_mini_cluster.cc:1428] Setting key c2fd513e012c915871f059397ff817ac
I20260806 22:40:24.050081 9619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:24.051748 9619 catalog_manager.cc:1347] Generated new cluster ID: b117e06ac6244bc09bb3abf74a47c139
I20260806 22:40:24.051800 9619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:24.062556 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.050434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.0.186.254:35233)
0806 22:40:24.050812 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.050821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.050855 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:40:24.050975 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.050983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.051040 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.052137 (+ 1097us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.052242 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.053669 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.054771 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.055459 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.055890 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.058362 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.058367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.058367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.058379 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.060321 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.060736 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.060738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.060739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.060844 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.061049 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.061050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.061050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.061340 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.062079 (+ 739us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.062157 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.062301 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":91,"threads_started":1}
I20260806 22:40:24.078554 9619 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:24.079308 9619 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:24.086108 9619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2156ee2c16e4f7180d8d33c329925dd: Generated new TSK 0
I20260806 22:40:24.086290 9619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:24.113736 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:24.194936 9640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:24.195142 9640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:24.195163 9640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:24.196260 9640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:24.196295 9640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:24.196318 9640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:24.197692 9640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:24.197865 9640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:24.198032 9640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:24.202781 9658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.202777 9653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.202782 9654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.203071 9640 server_base.cc:1034] running on GCE node
I20260806 22:40:24.203330 9640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:24.203563 9640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:24.204705 9640 hybrid_clock.cc:648] HybridClock initialized: now 1786056024204686 us; error 36 us; skew 500 ppm
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.210426 9640 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:24.210829 9640 webserver.cc:460] Webserver started at http://127.0.186.193:40807/ using document root <none> and password file <none>
I20260806 22:40:24.211015 9640 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:24.211054 9640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:24.211146 9640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:24.212194 9640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "f4e957bd5e9e432ba7c31c52d97f2817"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "19786da9f7306318af5c6cc03db7b2cf"
server_key_iv: "e78906fc56dbc3169d0a200a0905708d"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.212496 9640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "f4e957bd5e9e432ba7c31c52d97f2817"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "19786da9f7306318af5c6cc03db7b2cf"
server_key_iv: "e78906fc56dbc3169d0a200a0905708d"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.213718 9640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:24.214423 9670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.214596 9640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:24.214679 9640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "f4e957bd5e9e432ba7c31c52d97f2817"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "19786da9f7306318af5c6cc03db7b2cf"
server_key_iv: "e78906fc56dbc3169d0a200a0905708d"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.214735 9640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:24.228274 9640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:24.230444 9640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:24.230561 9640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:24.230758 9640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:24.231040 9640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:24.231071 9640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.231092 9640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:24.231103 9640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.236599 9640 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:36537
I20260806 22:40:24.236645 9848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:36537 every 8 connection(s)
I20260806 22:40:24.236982 9640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:24.238989 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 9640
I20260806 22:40:24.239089 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:24.239307 747 external_mini_cluster.cc:1428] Setting key 33524783dd1a4932857646ea179d98e5
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:24.246860 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.237748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:47333 (local address 127.0.186.254:35233)
0806 22:40:24.237821 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.237823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.237980 (+ 157us) server_negotiation.cc:407] Connection header received
0806 22:40:24.238949 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.238950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.238963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.239037 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.239762 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.240260 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.241286 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.241516 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.241573 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.243748 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.243750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.243750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.243760 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.245393 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.245813 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.245814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.245814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.245883 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.246128 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.246130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.246130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.246278 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.246360 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.246669 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.246762 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:40:24.246994 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.237677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35233 (local address 127.0.186.193:47333)
0806 22:40:24.237853 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:24.237894 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:24.238847 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:24.239129 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:24.239134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:24.239463 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:24.239663 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.239669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.240346 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.240347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.241196 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.241199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.241590 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.241591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.241717 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.242213 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:24.242221 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:24.243651 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:24.245492 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.245505 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.245507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.245735 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.245958 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.245959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.245959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.246067 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.246383 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:24.246390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:24.246521 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:40:24.246628 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:40:24.246756 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260806 22:40:24.247668 9849 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35233
I20260806 22:40:24.247766 9849 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:24.247956 9849 heartbeater.cc:511] Master 127.0.186.254:35233 requested a full tablet report, sending...
I20260806 22:40:24.248580 9548 ts_manager.cc:194] Registered new tserver with Master: f4e957bd5e9e432ba7c31c52d97f2817 (127.0.186.193:36537)
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
I20260806 22:40:24.249457 9548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:47333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:24.267572 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:24.349876 9856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:24.350085 9856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:24.350111 9856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:24.351248 9856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:24.351310 9856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:24.351336 9856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:24.352823 9856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:24.353049 9856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:24.353257 9856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:24.357858 9865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.357885 9863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.357941 9856 server_base.cc:1034] running on GCE node
W20260806 22:40:24.357899 9868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.358316 9856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:24.358539 9856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:24.359700 9856 hybrid_clock.cc:648] HybridClock initialized: now 1786056024359681 us; error 28 us; skew 500 ppm
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.362344 9856 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:24.362748 9856 webserver.cc:460] Webserver started at http://127.0.186.194:45487/ using document root <none> and password file <none>
I20260806 22:40:24.362939 9856 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:24.362982 9856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:24.363070 9856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:24.364079 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "9263b0beff6d48d5b3d08646ded6cb21"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "f994c78a7957013b8247ff3c8d7684f2"
server_key_iv: "8a3b334fa74766a4158a91917bbabc47"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.364367 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "9263b0beff6d48d5b3d08646ded6cb21"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "f994c78a7957013b8247ff3c8d7684f2"
server_key_iv: "8a3b334fa74766a4158a91917bbabc47"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.365546 9856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:24.366220 9881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.366367 9856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:24.366452 9856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "9263b0beff6d48d5b3d08646ded6cb21"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "f994c78a7957013b8247ff3c8d7684f2"
server_key_iv: "8a3b334fa74766a4158a91917bbabc47"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.366509 9856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:24.376943 9856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:24.378695 9856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:24.378799 9856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:24.379000 9856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:24.379285 9856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:24.379329 9856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.379351 9856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:24.379366 9856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.384758 9856 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:38959
I20260806 22:40:24.384826 10047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:38959 every 8 connection(s)
I20260806 22:40:24.385082 9856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:24.392808 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 9856
I20260806 22:40:24.392899 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:24.393102 747 external_mini_cluster.cc:1428] Setting key d3beeda0537d2b11a86dd516a75caed8
I20260806 22:40:24.394189 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.385758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:37721 (local address 127.0.186.254:35233)
0806 22:40:24.385808 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.385810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.385989 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:40:24.387113 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.387115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.387129 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.387181 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.387804 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.388308 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.389274 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.389480 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.389537 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.391635 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.391637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.391637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.391645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.392763 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.393206 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.393207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.393207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.393278 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.393519 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.393520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.393520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.393660 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.393726 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.394044 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.394125 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:24.394398 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.385723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35233 (local address 127.0.186.194:37721)
0806 22:40:24.385866 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:24.385905 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:24.386972 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:24.387251 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:24.387255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:24.387525 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:24.387704 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.387709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.388393 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.388394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.389187 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.389190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.389550 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.389551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.389683 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.390138 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:24.390146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:24.391546 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:24.392837 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.392845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.392847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.393121 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.393348 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.393349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.393349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.393466 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.393728 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:24.393736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:24.393890 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:40:24.394023 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 22:40:24.394198 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260806 22:40:24.394846 10048 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35233
I20260806 22:40:24.394968 10048 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:24.395143 10048 heartbeater.cc:511] Master 127.0.186.254:35233 requested a full tablet report, sending...
I20260806 22:40:24.395538 9548 ts_manager.cc:194] Registered new tserver with Master: 9263b0beff6d48d5b3d08646ded6cb21 (127.0.186.194:38959)
I20260806 22:40:24.396001 9548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:37721
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:24.418286 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:24.498287 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:24.498484 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:24.498507 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:24.499639 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:24.499680 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:24.499706 10055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:24.501118 10055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:24.501292 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:24.501468 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:24.506296 10055 server_base.cc:1034] running on GCE node
W20260806 22:40:24.506253 10061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.506263 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.506263 10070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.506716 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:24.506950 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:24.508116 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786056024508092 us; error 32 us; skew 500 ppm
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.510502 10055 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:24.510882 10055 webserver.cc:460] Webserver started at http://127.0.186.195:40177/ using document root <none> and password file <none>
I20260806 22:40:24.511062 10055 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:24.511101 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:24.511189 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:24.512228 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "da8b5e9a07c543d0b7f1adc3069da5cf"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "622db44db9ac4862f85848fdaeee4fe3"
server_key_iv: "f59f839d82e4e23cbdd17eb9d7ee52c9"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.512547 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "da8b5e9a07c543d0b7f1adc3069da5cf"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "622db44db9ac4862f85848fdaeee4fe3"
server_key_iv: "f59f839d82e4e23cbdd17eb9d7ee52c9"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.513789 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:24.514477 10083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.514605 10055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:24.514681 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "da8b5e9a07c543d0b7f1adc3069da5cf"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-gjrz"
server_key: "622db44db9ac4862f85848fdaeee4fe3"
server_key_iv: "f59f839d82e4e23cbdd17eb9d7ee52c9"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.514745 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:24.531919 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:24.533696 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:24.533807 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:24.534027 10055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:24.534313 10055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:24.534348 10055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.534368 10055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:24.534391 10055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.539745 10055 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:37083
I20260806 22:40:24.539788 10263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:37083 every 8 connection(s)
I20260806 22:40:24.540086 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:24.543174 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 10055
I20260806 22:40:24.543285 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:24.543467 747 external_mini_cluster.cc:1428] Setting key 48079e6793866248d27262d784c465c9
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:24.548688 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.540725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:52491 (local address 127.0.186.254:35233)
0806 22:40:24.540787 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.540789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.540968 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:40:24.541886 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.541887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.541900 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.541953 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.542549 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.543052 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.544254 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.544501 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.544554 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.546421 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.546423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.546423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.546431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.547512 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.547878 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.547879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.547879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.547943 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.548128 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.548129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.548130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.548243 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.548290 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.548539 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.548606 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:40:24.548875 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.540695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35233 (local address 127.0.186.195:52491)
0806 22:40:24.540854 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:24.540889 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:24.541769 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:24.542028 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:24.542032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:24.542268 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:24.542456 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.542460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.543137 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.543139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.544178 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:24.544181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.544581 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.544582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:24.544706 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.545075 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:24.545083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:24.546336 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:24.547598 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.547605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.547607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.547805 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.547993 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:24.547994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:24.547994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:24.548077 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:24.548289 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:24.548294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:24.548433 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:40:24.548525 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:40:24.548679 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260806 22:40:24.549245 10264 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35233
I20260806 22:40:24.549331 10264 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:24.549479 10264 heartbeater.cc:511] Master 127.0.186.254:35233 requested a full tablet report, sending...
I20260806 22:40:24.549832 9548 ts_manager.cc:194] Registered new tserver with Master: da8b5e9a07c543d0b7f1adc3069da5cf (127.0.186.195:37083)
I20260806 22:40:24.550166 9548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:52491
I20260806 22:40:24.554998 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/test-admin.keytab.
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.571194 747 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:40:24.572878 10273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:24.572067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.0.186.193:36537)
0806 22:40:24.572226 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.572231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.572239 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:24.572263 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.572265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.572278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.572341 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.572797 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44648: BlockingRecv error: recv got EOF from 127.0.0.1:44648 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260806 22:40:24.572934 747 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.573800 747 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.574642 747 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:24.581468 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.575112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.0.186.193:36537)
0806 22:40:24.575162 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.575164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.575176 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:24.575231 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.575232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.575242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.575323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.575677 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.576383 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.577076 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.577372 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.577418 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.579403 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.579411 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.579411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.579418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.580621 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.580895 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.580897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.580897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.580947 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.581066 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.581067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.581067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.581176 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.581264 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.581320 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.581396 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:40:24.583004 10273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:24.582547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.0.186.193:36537)
0806 22:40:24.582607 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.582608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.582616 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:24.582661 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.582661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.582669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.582711 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.582960 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44672: BlockingRecv error: recv got EOF from 127.0.0.1:44672 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:40:24.583103 747 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.584002 747 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.584746 747 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:24.591040 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.585211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.0.186.193:36537)
0806 22:40:24.585257 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.585259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.585267 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:24.585310 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.585311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.585320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.585363 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.585694 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.586180 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.586883 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.587091 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.587145 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.588970 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.588971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.588972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.588979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.590063 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.590375 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.590376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.590376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.590443 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.590601 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.590602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.590602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.590693 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.590772 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.590877 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.590954 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:40:24.592440 10273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:24.591967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.0.186.193:36537)
0806 22:40:24.592011 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.592013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.592036 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:40:24.592100 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.592101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.592109 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.592151 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.592390 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44682: BlockingRecv error: recv got EOF from 127.0.0.1:44682 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 22:40:24.592497 747 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055997241161-747-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.593323 747 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.594086 747 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[9475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.193@KRBTEST.COM
I20260806 22:40:24.600030 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.594531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.0.186.193:36537)
0806 22:40:24.594578 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.594579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.594587 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:24.594660 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.594661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.594669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.594708 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.595058 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.595573 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.596206 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.596382 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.596440 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.598139 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.598140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.598141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.598147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.599108 (+ 961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.599472 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.599473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.599473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.599532 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.599710 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.599711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.599711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.599791 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.599851 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.599923 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.599977 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:24.600575 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 9640
I20260806 22:40:24.611724 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 9856
I20260806 22:40:24.636050 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 10055
I20260806 22:40:24.643182 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 9497
2026-08-06T22:40:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2017 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[10298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[10298](info): set up 2 sockets
Aug 06 22:40:24 dist-test-slave-gjrz krb5kdc[10298](info): commencing operation
krb5kdc: starting...
W20260806 22:40:25.770479 747 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:25 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab.
Entry for principal oryx/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab.
I20260806 22:40:25.884012 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46337
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab
--principal=oryx/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:25.964576 10316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:25.964799 10316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:25.964833 10316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:25.966033 10316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:25.966076 10316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:25.966094 10316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:25.966111 10316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:25.966125 10316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:25.967675 10316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab
--principal=oryx/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:25.967921 10316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:25.968155 10316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:25.973177 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:25.973215 10316 server_base.cc:1034] running on GCE node
W20260806 22:40:25.973209 10332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:25.973186 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:25.973697 10316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:25.973989 10316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:25.975145 10316 hybrid_clock.cc:648] HybridClock initialized: now 1786056025975131 us; error 28 us; skew 500 ppm
Aug 06 22:40:25 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:25.977643 10316 init.cc:377] Logged in from keytab as oryx/127.0.186.254@KRBTEST.COM (short username oryx)
I20260806 22:40:25.978116 10316 webserver.cc:460] Webserver started at http://127.0.186.254:42695/ using document root <none> and password file <none>
I20260806 22:40:25.978343 10316 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:25.978394 10316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:25.978525 10316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:25.979758 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "c4ad2cbb63fc4481ad69570692339466"
format_stamp: "Formatted at 2026-08-06 22:40:25 on dist-test-slave-gjrz"
server_key: "f0e58b4bab882f93cca3943a0dea9dc8"
server_key_iv: "c7c87dd380e271b32a0065082ac953e2"
server_key_version: "encryptionkey@0"
I20260806 22:40:25.980187 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "c4ad2cbb63fc4481ad69570692339466"
format_stamp: "Formatted at 2026-08-06 22:40:25 on dist-test-slave-gjrz"
server_key: "f0e58b4bab882f93cca3943a0dea9dc8"
server_key_iv: "c7c87dd380e271b32a0065082ac953e2"
server_key_version: "encryptionkey@0"
I20260806 22:40:25.981586 10316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:25.982307 10342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:25.982476 10316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:25.982564 10316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "c4ad2cbb63fc4481ad69570692339466"
format_stamp: "Formatted at 2026-08-06 22:40:25 on dist-test-slave-gjrz"
server_key: "f0e58b4bab882f93cca3943a0dea9dc8"
server_key_iv: "c7c87dd380e271b32a0065082ac953e2"
server_key_version: "encryptionkey@0"
I20260806 22:40:25.982636 10316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.002346 10316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.004462 10316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:26.004612 10316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.008786 10316 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:46337
I20260806 22:40:26.008843 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:46337 every 8 connection(s)
I20260806 22:40:26.009202 10316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:26.010221 10415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:26.012645 10415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Bootstrap starting.
I20260806 22:40:26.013265 10415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:26.013545 10415 log.cc:824] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:26.014317 10415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: No bootstrap required, opened a new log
I20260806 22:40:26.015974 10415 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } }
I20260806 22:40:26.016165 10415 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:26.016188 10415 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ad2cbb63fc4481ad69570692339466, State: Initialized, Role: FOLLOWER
I20260806 22:40:26.016327 10415 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } }
I20260806 22:40:26.016402 10415 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:26.016425 10415 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:26.016455 10415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:26.017336 10415 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } }
I20260806 22:40:26.017438 10415 leader_election.cc:302] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4ad2cbb63fc4481ad69570692339466; no voters:
I20260806 22:40:26.017608 10415 leader_election.cc:288] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:26.017658 10419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:26.017848 10419 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 LEADER]: Becoming Leader. State: Replica: c4ad2cbb63fc4481ad69570692339466, State: Running, Role: LEADER
I20260806 22:40:26.017899 10415 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:26.017963 10419 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } }
I20260806 22:40:26.018391 10424 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ad2cbb63fc4481ad69570692339466. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } }
I20260806 22:40:26.018499 10424 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:26.018865 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 10316
I20260806 22:40:26.018971 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:26.019189 747 external_mini_cluster.cc:1428] Setting key dacfa16181a205b9e689be1027c0b7e2
I20260806 22:40:26.018707 10434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:26.018661 10421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } }
I20260806 22:40:26.019562 10421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:26.019984 10434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:26.021535 10434 catalog_manager.cc:1347] Generated new cluster ID: 37389e40cc1145b1a630b0c6ba518290
I20260806 22:40:26.021593 10434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:26.032158 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.019731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.0.186.254:46337)
0806 22:40:26.023401 (+ 3670us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.023410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.023441 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:40:26.023552 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.023558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.023601 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.023740 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.024410 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.025365 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.026095 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.026827 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.026932 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.029182 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.029186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.029186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.029198 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.030723 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.031045 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.031048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.031048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.031135 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.031333 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.031334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.031334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.031574 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.031732 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.031838 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.031977 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3615,"thread_start_us":71,"threads_started":1}
I20260806 22:40:26.042364 10434 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:26.043087 10434 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:26.052321 10434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Generated new TSK 0
I20260806 22:40:26.052584 10434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
I20260806 22:40:26.081645 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:26.166307 10454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:26.166527 10454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:26.166550 10454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:26.167750 10454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:26.167793 10454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:26.167819 10454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:26.169262 10454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:26.169492 10454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:26.169715 10454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.174465 10466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.174480 10462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.174526 10468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.174638 10454 server_base.cc:1034] running on GCE node
I20260806 22:40:26.174876 10454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.175109 10454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.176262 10454 hybrid_clock.cc:648] HybridClock initialized: now 1786056026176239 us; error 36 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:26.178941 10454 init.cc:377] Logged in from keytab as oryx/127.0.186.193@KRBTEST.COM (short username oryx)
I20260806 22:40:26.179492 10454 webserver.cc:460] Webserver started at http://127.0.186.193:46819/ using document root <none> and password file <none>
I20260806 22:40:26.179737 10454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.179787 10454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.179913 10454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:26.180969 10454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "308bc0d3695040e6a296bbb4613fea1d"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "a89971886170658a9ace4f35c813dea7"
server_key_iv: "dcd732129bb2a98ff2de480dbad54a1a"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.181316 10454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "308bc0d3695040e6a296bbb4613fea1d"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "a89971886170658a9ace4f35c813dea7"
server_key_iv: "dcd732129bb2a98ff2de480dbad54a1a"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.182605 10454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:26.183364 10481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.183521 10454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:26.183606 10454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "308bc0d3695040e6a296bbb4613fea1d"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "a89971886170658a9ace4f35c813dea7"
server_key_iv: "dcd732129bb2a98ff2de480dbad54a1a"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.183671 10454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.202626 10454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.204541 10454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:26.204684 10454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.204911 10454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:26.205240 10454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:26.205276 10454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.205305 10454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:26.205329 10454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.210898 10454 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:35121
I20260806 22:40:26.210960 10660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:35121 every 8 connection(s)
I20260806 22:40:26.211287 10454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:26.217620 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 10454
I20260806 22:40:26.217728 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:26.217927 747 external_mini_cluster.cc:1428] Setting key 82b35ba24b5a4fa0b0e4651fe239f48d
I20260806 22:40:26.220609 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.211998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54001 (local address 127.0.186.254:46337)
0806 22:40:26.212054 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.212056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.212245 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:40:26.212715 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.212716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.212726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.212778 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.213692 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.214191 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.215144 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.215370 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.215422 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.217524 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.217525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.217526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.217533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.219094 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.219622 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.219623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.219624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.219682 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.219929 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.219930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.219930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.220048 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.220094 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.220440 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.220537 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:26.221043 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.211963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:54001)
0806 22:40:26.212132 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.212168 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.212633 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.212846 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.212849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.213424 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.213620 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.213625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.214273 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.214274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.215065 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.215068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.215466 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.215467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.215589 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.216040 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.216049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.217428 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.219173 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.219181 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.219183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.219547 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.219739 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.219740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.219741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.219872 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.220098 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.220102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.220265 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.220703 (+ 438us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.220871 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260806 22:40:26.221556 10661 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:26.221642 10661 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:26.221814 10661 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:26.222399 10365 ts_manager.cc:194] Registered new tserver with Master: 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
I20260806 22:40:26.223287 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:54001
WARNING: no policy specified for oryx/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
I20260806 22:40:26.244295 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:26.327647 10669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:26.327853 10669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:26.327876 10669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:26.328989 10669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:26.329021 10669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:26.329046 10669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:26.330446 10669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:26.330638 10669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:26.330817 10669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.335608 10680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.335597 10677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.335583 10678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.335742 10669 server_base.cc:1034] running on GCE node
I20260806 22:40:26.335945 10669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.336167 10669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.337323 10669 hybrid_clock.cc:648] HybridClock initialized: now 1786056026337307 us; error 30 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:26.339819 10669 init.cc:377] Logged in from keytab as oryx/127.0.186.194@KRBTEST.COM (short username oryx)
I20260806 22:40:26.340292 10669 webserver.cc:460] Webserver started at http://127.0.186.194:36259/ using document root <none> and password file <none>
I20260806 22:40:26.340514 10669 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.340566 10669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.340687 10669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:26.341722 10669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "91f860820c864aef8989d22e2e64aa62"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "ae63ef3ce5890e6616a10ea3606d223a"
server_key_iv: "c073d15161fb9d89af478770a836c9b4"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.342082 10669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "91f860820c864aef8989d22e2e64aa62"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "ae63ef3ce5890e6616a10ea3606d223a"
server_key_iv: "c073d15161fb9d89af478770a836c9b4"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.343405 10669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:26.344135 10691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.344296 10669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:26.344389 10669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "91f860820c864aef8989d22e2e64aa62"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "ae63ef3ce5890e6616a10ea3606d223a"
server_key_iv: "c073d15161fb9d89af478770a836c9b4"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.344461 10669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.358799 10669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.360538 10669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:26.360673 10669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.360872 10669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:26.361162 10669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:26.361195 10669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.361223 10669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:26.361248 10669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.366791 10669 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:40993
I20260806 22:40:26.366853 10883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:40993 every 8 connection(s)
I20260806 22:40:26.367152 10669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:26.369686 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 10669
I20260806 22:40:26.369786 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:26.369971 747 external_mini_cluster.cc:1428] Setting key 8449c516cfa3244c3c8b24894a470810
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:26.376256 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.367894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:59209 (local address 127.0.186.254:46337)
0806 22:40:26.367973 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.367975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.368146 (+ 171us) server_negotiation.cc:407] Connection header received
0806 22:40:26.368671 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.368673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.368685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.368737 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.369673 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.370306 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.371350 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.371556 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.371614 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.373839 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.373841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.373841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.373849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.374994 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.375389 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.375390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.375391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.375446 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.375664 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.375665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.375665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.375771 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.375826 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.376081 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.376168 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:40:26.376430 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.367817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:59209)
0806 22:40:26.368020 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.368065 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.368567 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.368807 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.368811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.369372 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.369573 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.369578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.370391 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.370392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.371236 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.371239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.371626 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.371627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.371758 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.372226 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.372234 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.373750 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.375067 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.375075 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.375077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.375324 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.375503 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.375503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.375504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.375608 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.375826 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.375833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.375965 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.376060 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.376215 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260806 22:40:26.376874 10884 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:26.376986 10884 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:26.377175 10884 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:26.377548 10365 ts_manager.cc:194] Registered new tserver with Master: 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
I20260806 22:40:26.377911 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:59209
WARNING: no policy specified for oryx/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
I20260806 22:40:26.397712 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:26.479537 10892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:26.479744 10892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:26.479775 10892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:26.480902 10892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:26.480950 10892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:26.480985 10892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:26.482432 10892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:26.482673 10892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:26.482884 10892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.487648 10901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.487722 10904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.487705 10902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.487823 10892 server_base.cc:1034] running on GCE node
I20260806 22:40:26.488008 10892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.488231 10892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.489387 10892 hybrid_clock.cc:648] HybridClock initialized: now 1786056026489358 us; error 30 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:26.491952 10892 init.cc:377] Logged in from keytab as oryx/127.0.186.195@KRBTEST.COM (short username oryx)
I20260806 22:40:26.492396 10892 webserver.cc:460] Webserver started at http://127.0.186.195:37791/ using document root <none> and password file <none>
I20260806 22:40:26.492619 10892 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.492666 10892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.492787 10892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:26.493796 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "8633da54eb634804b8ed726e4eee00df"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "163aa033dfb4a331d781495681466195"
server_key_iv: "77659545379af786b49839b214c66e92"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.494123 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "8633da54eb634804b8ed726e4eee00df"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "163aa033dfb4a331d781495681466195"
server_key_iv: "77659545379af786b49839b214c66e92"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.495378 10892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:40:26.496068 10916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.496223 10892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:26.496308 10892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "8633da54eb634804b8ed726e4eee00df"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "163aa033dfb4a331d781495681466195"
server_key_iv: "77659545379af786b49839b214c66e92"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.496388 10892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.519990 10892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.521952 10892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:26.522090 10892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.522302 10892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:26.522622 10892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:26.522658 10892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.522686 10892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:26.522710 10892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.528285 10892 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:44893
I20260806 22:40:26.528343 11083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:44893 every 8 connection(s)
I20260806 22:40:26.528646 10892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:26.533102 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 10892
I20260806 22:40:26.533183 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:26.533373 747 external_mini_cluster.cc:1428] Setting key 3c108a19f59e891bfdab637cab6c4bbf
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:26.537266 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.529291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:42061 (local address 127.0.186.254:46337)
0806 22:40:26.529368 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.529370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.529539 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:40:26.529988 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.529989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.529999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.530049 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.530935 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.531496 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.532645 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.532832 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.532884 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.534941 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.534942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.534942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.534949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.536177 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.536571 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.536572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.536572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.536620 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.536799 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.536800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.536800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.536893 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.536935 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.537146 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.537204 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:40:26.537448 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.529256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:42061)
0806 22:40:26.529421 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.529457 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.529917 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.530105 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.530108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.530667 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.530861 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.530867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.531576 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.531576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.532568 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.532571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.532893 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.532894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.532996 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.533496 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.533505 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.534856 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.536261 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.536267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.536269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.536506 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.536663 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.536663 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.536664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.536757 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.536933 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.536936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.537049 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.537126 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.537243 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260806 22:40:26.537905 11084 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:26.537992 11084 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:26.538166 11084 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:26.538491 10365 ts_manager.cc:194] Registered new tserver with Master: 8633da54eb634804b8ed726e4eee00df (127.0.186.195:44893)
I20260806 22:40:26.538821 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:42061
I20260806 22:40:26.545035 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.253.keytab.
Entry for principal oryx/127.0.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.253.keytab.
I20260806 22:40:26.569633 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:40433
--webserver_interface=127.0.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.253.keytab
--principal=oryx/127.0.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46337,127.0.186.253:40433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:26.658263 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:26.658474 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:26.658497 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:26.659770 11092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:26.659811 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:26.659824 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:26.659835 11092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:26.659859 11092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:26.661394 11092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46337,127.0.186.253:40433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.253.keytab
--principal=oryx/127.0.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:40433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
W20260806 22:40:26.661513 11092 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.186.254:46337,127.0.186.253:40433'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:40:26.661630 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:26.661808 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.667479 11099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.667484 11101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.667517 11098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.668066 11092 server_base.cc:1034] running on GCE node
I20260806 22:40:26.668280 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.668541 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.669673 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786056026669655 us; error 35 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:26.672201 11092 init.cc:377] Logged in from keytab as oryx/127.0.186.253@KRBTEST.COM (short username oryx)
I20260806 22:40:26.672624 11092 webserver.cc:460] Webserver started at http://127.0.186.253:45859/ using document root <none> and password file <none>
I20260806 22:40:26.672825 11092 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.672868 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.672991 11092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:26.674093 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/instance:
uuid: "298726ce9e304e24bca54ff4f3c82854"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "e1482a736faa43aa1dc5a7ee673a99d4"
server_key_iv: "769faf11820a8d9bad0c5e170fc577ce"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.674422 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal/instance:
uuid: "298726ce9e304e24bca54ff4f3c82854"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "e1482a736faa43aa1dc5a7ee673a99d4"
server_key_iv: "769faf11820a8d9bad0c5e170fc577ce"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.675796 11092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:40:26.676626 11112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.676774 11092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:26.676862 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "298726ce9e304e24bca54ff4f3c82854"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "e1482a736faa43aa1dc5a7ee673a99d4"
server_key_iv: "769faf11820a8d9bad0c5e170fc577ce"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.676926 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.690440 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.692440 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:26.692550 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.698668 11092 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.253:40433
I20260806 22:40:26.698827 11186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.253:40433 every 8 connection(s)
I20260806 22:40:26.699138 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
I20260806 22:40:26.700253 11187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:26.702490 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:26.704865 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 11092
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.253@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
I20260806 22:40:26.713001 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.703203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.0.186.254:46337)
0806 22:40:26.703300 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.703302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.703543 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:40:26.704008 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.704009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.704023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.704092 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.704710 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.705368 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.706755 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.707089 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.707171 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.709624 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.709626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.709626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.709633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.711483 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.711959 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.711960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.711960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.712028 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.712322 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.712322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.712323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.712470 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.712566 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.712868 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.712936 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:26.713244 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.703158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44708)
0806 22:40:26.703409 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.703445 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.703914 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.704160 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.704163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.704405 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.704601 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.704605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.705472 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.705474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.706651 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.706654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.707172 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.707173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.707393 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.707991 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.708000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.709538 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.711550 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.711556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.711558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.711881 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.712098 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.712099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.712099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.712240 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.712566 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.712571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.712719 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.712843 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.713004 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":181,"threads_started":1}
I20260806 22:40:26.713994 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:26.715050 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.705390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.0.186.253:40433)
0806 22:40:26.705578 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.705582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.705598 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:40:26.705618 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.705619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.705635 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.705698 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.706442 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.707675 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.708434 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.708874 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.708941 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.711435 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.711442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.711442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.711449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.713559 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.713939 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.713942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.713942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.714006 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.714201 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.714202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.714202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.714393 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.714526 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.714823 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.714897 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.253@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
I20260806 22:40:26.721321 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.714234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47090)
0806 22:40:26.714272 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.714281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.714314 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.714632 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.714633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.714734 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.714802 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.714804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.715699 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.715700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.716184 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.716186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.716551 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.716551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.716678 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.716986 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.716990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.718177 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.720457 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.720458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.720458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.720614 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.720772 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.720772 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.720772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.720824 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.721025 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.721026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.721071 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.721129 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.721222 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 22:40:26.721570 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.714269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.0.186.253:40433)
0806 22:40:26.714492 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.714493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.714501 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:26.714517 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.714518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.714527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.714628 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.714879 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.715621 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.716251 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.716493 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.716554 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.718249 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.718250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.718250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.718256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.720383 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.720672 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.720672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.720673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.720720 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.720873 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.720874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.720874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.720965 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.721019 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.721155 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.721272 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
I20260806 22:40:26.722911 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Bootstrap starting.
I20260806 22:40:26.723942 11187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:26.724557 11187 log.cc:824] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:26.725324 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: No bootstrap required, opened a new log
I20260806 22:40:26.727733 11187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } }
I20260806 22:40:26.727856 11187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:26.727877 11187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298726ce9e304e24bca54ff4f3c82854, State: Initialized, Role: FOLLOWER
I20260806 22:40:26.727993 11187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } }
I20260806 22:40:26.728300 11187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:26.728310 11198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } }
I20260806 22:40:26.728447 11198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:40:26.730876 11211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:40:26.730978 11211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:40:26.732191 11092 master_runner.cc:428] Detected that this master 298726ce9e304e24bca54ff4f3c82854 is joining an existing cluster
I20260806 22:40:26.732259 11092 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.186.253:40433
I20260806 22:40:26.740973 11222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.733681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47096)
0806 22:40:26.733877 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.733885 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.733943 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.734108 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.734109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.734452 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.734645 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.734647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.735280 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.735281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.736509 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.736512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.736958 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.737088 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.737680 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.737684 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.738123 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.739802 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.739803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.739804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.740006 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.740178 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.740179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.740232 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.740524 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.740525 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.740618 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.740712 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.740818 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260806 22:40:26.740983 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.733783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.0.186.253:40433)
0806 22:40:26.733816 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.733818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.733926 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:40:26.734001 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.734001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.734009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.734031 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.734730 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.735205 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736601 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.736835 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736866 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.738212 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.738213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.738214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.738219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.739731 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740077 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.740078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.740078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.740126 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740314 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.740316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.740316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.740457 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.740532 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.740742 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.740891 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":20}
I20260806 22:40:26.742065 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.733315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44720)
0806 22:40:26.733837 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.733846 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.733932 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.734086 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.734087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.734440 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.734645 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.734647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.735611 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.735611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.736527 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.736528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.736987 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.737091 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.737780 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.737785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.738168 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.740263 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.740264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.740420 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.740677 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.740678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.740748 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.741696 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.741696 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.741742 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.741849 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.741979 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":3,"thread_start_us":238,"threads_started":1}
I20260806 22:40:26.742089 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.733341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.0.186.254:46337)
0806 22:40:26.733380 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.733382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.733904 (+ 522us) server_negotiation.cc:407] Connection header received
0806 22:40:26.733996 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.733997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.734009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.734078 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.734732 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.735516 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736610 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.736911 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.736983 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.738250 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.738252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.738252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.738261 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.740179 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740484 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.740485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.740485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.740601 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.740817 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.740818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.740818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.740993 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.741666 (+ 673us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.741913 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.742001 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.254@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
I20260806 22:40:26.751590 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.744367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.0.186.253:40433)
0806 22:40:26.744406 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.744408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.744545 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:40:26.744612 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.744613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.744624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.744686 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.745126 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.745870 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.746660 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.746955 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.747030 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.749178 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.749180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.749180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.749187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.750513 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.750865 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.750866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.750867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.750932 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.751090 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.751090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.751090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.751204 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.751284 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.751440 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.751512 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:26.751642 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.744305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47110)
0806 22:40:26.744444 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.744486 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.744558 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.744722 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.744730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.744996 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.745046 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.745048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.745954 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.745955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.746523 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.746525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.747028 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.747029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.747114 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.747523 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.747527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.749073 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.750594 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.750601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.750603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.750785 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.750991 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.750991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.750991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.751044 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.751285 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.751291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.751336 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.751404 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.751567 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260806 22:40:26.752395 10365 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.186.253:40433: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "298726ce9e304e24bca54ff4f3c82854"
member_type: NON_VOTER
last_known_addr {
host: "127.0.186.253"
port: 40433
}
attrs {
promote: true
}
}
dest_uuid: "c4ad2cbb63fc4481ad69570692339466"
cas_config_opid_index: -1
I20260806 22:40:26.752789 10365 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } }
I20260806 22:40:26.753299 11225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:26.753731 11151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:26.754558 11225 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.186.253:40433
I20260806 22:40:26.754722 11224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:26.754805 11224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:26.754848 11092 master.cc:537] Master@127.0.186.253:40433 shutting down...
I20260806 22:40:26.755044 11151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:40:26.755374 11224 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:40:26.756096 10350 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 22:40:26.756843 10350 consensus_peers.cc:487] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 -> Peer 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253:40433): Couldn't send request to peer 298726ce9e304e24bca54ff4f3c82854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:40:26.758263 11092 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:26.758354 11092 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:26.758401 11092 tablet_replica.cc:330] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: stopping tablet replica
I20260806 22:40:26.760654 11092 master.cc:552] Master@127.0.186.253:40433 shutdown complete.
I20260806 22:40:26.760692 11092 master_runner.cc:305] Clearing existing system tablet
I20260806 22:40:26.760816 11092 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.760855 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.761395 11092 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:26.761852 11235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.761987 11092 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.762049 11092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.762074 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "298726ce9e304e24bca54ff4f3c82854"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "e1482a736faa43aa1dc5a7ee673a99d4"
server_key_iv: "769faf11820a8d9bad0c5e170fc577ce"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.762749 11092 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:26.764096 11092 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:40:26.764156 11092 log.cc:1196] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:40:26.764324 11092 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Deleting consensus metadata
I20260806 22:40:26.764432 11092 master_runner.cc:315] Copying system tablet from 127.0.186.254:46337
I20260806 22:40:26.764966 11092 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.254:46337
I20260806 22:40:26.771049 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.765370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.0.186.254:46337)
0806 22:40:26.765418 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.765420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.765482 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:40:26.765554 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.765555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.765563 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.765607 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.766000 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.766551 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.767569 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.767752 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.767802 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.768733 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.768734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.768734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.768741 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.769969 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.770355 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.770356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.770356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.770418 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.770625 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.770626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.770627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.770739 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.770798 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.770923 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.770983 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:40:26.771121 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.765295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44724)
0806 22:40:26.765413 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:26.765425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:26.765498 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:26.765625 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:26.765626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:26.765860 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:26.765919 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.765921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.766621 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.766622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.767496 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:26.767499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.767814 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.767816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:26.767879 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.768264 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:26.768269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:26.768648 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:26.770040 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.770041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.770041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.770254 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.770488 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:26.770488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:26.770489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:26.770555 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:26.770793 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:26.770794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:26.770839 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:40:26.770908 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:40:26.771015 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260806 22:40:26.771530 10395 tablet_copy_service.cc:140] P c4ad2cbb63fc4481ad69570692339466: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 298726ce9e304e24bca54ff4f3c82854 ({username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:44724)
I20260806 22:40:26.771641 10395 tablet_copy_service.cc:161] P c4ad2cbb63fc4481ad69570692339466: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 298726ce9e304e24bca54ff4f3c82854 at {username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:44724: session id = 298726ce9e304e24bca54ff4f3c82854-00000000000000000000000000000000
I20260806 22:40:26.772557 10395 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:40:26.773209 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:40:26.774197 11092 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: tablet copy: Starting download of 0 data blocks...
I20260806 22:40:26.774331 11092 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: tablet copy: Starting download of 1 WAL segments...
I20260806 22:40:26.775354 11092 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:40:26.776114 10395 tablet_copy_service.cc:342] P c4ad2cbb63fc4481ad69570692339466: Request end of tablet copy session 298726ce9e304e24bca54ff4f3c82854-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:44724
I20260806 22:40:26.776161 10395 tablet_copy_service.cc:433] P c4ad2cbb63fc4481ad69570692339466: ending tablet copy session 298726ce9e304e24bca54ff4f3c82854-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 298726ce9e304e24bca54ff4f3c82854
I20260806 22:40:26.778611 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.779680 11246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.779762 11245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.779847 11248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.780040 11092 server_base.cc:1034] running on GCE node
I20260806 22:40:26.780126 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.780231 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.781333 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786056026781327 us; error 32 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:26.782634 11092 init.cc:377] Logged in from keytab as oryx/127.0.186.253@KRBTEST.COM (short username oryx)
I20260806 22:40:26.782887 11092 webserver.cc:460] Webserver started at http://127.0.186.253:45981/ using document root <none> and password file <none>
I20260806 22:40:26.782991 11092 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:26.783056 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:26.783720 11092 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:26.784174 11257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.784301 11092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:26.784344 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "298726ce9e304e24bca54ff4f3c82854"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "e1482a736faa43aa1dc5a7ee673a99d4"
server_key_iv: "769faf11820a8d9bad0c5e170fc577ce"
server_key_version: "encryptionkey@0"
I20260806 22:40:26.784400 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:26.810531 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:26.812064 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:26.817214 11092 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.253:40433
I20260806 22:40:26.817241 11340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.253:40433 every 8 connection(s)
I20260806 22:40:26.817525 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
I20260806 22:40:26.818524 11342 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:26.819051 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Bootstrap starting.
I20260806 22:40:26.822234 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:26.822484 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Bootstrap complete.
I20260806 22:40:26.822686 11342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } }
I20260806 22:40:26.822779 11342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 298726ce9e304e24bca54ff4f3c82854, State: Initialized, Role: LEARNER
I20260806 22:40:26.822827 11342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } }
I20260806 22:40:26.823019 11342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:26.822999 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:26.823133 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:40:26.824707 11362 catalog_manager.cc:1259] Loaded cluster ID: 37389e40cc1145b1a630b0c6ba518290
I20260806 22:40:26.824841 11362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: loading cluster ID for follower catalog manager: success
I20260806 22:40:26.825728 11362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: acquiring CA information for follower catalog manager: success
I20260806 22:40:26.825973 11362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:40:26.873904 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:26.867005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.0.186.253:40433)
0806 22:40:26.867160 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:26.867162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:26.867169 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:26.867191 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:26.867192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:26.867201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:26.867292 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:26.867893 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.868959 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.869687 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:26.870009 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:26.870081 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:26.871370 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:26.871372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:26.871372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:26.871380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:26.872823 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.873170 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.873171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.873172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.873231 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:26.873430 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:26.873431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:26.873431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:26.873561 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:26.873644 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:26.873721 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:40:26.873800 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.0.186.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.252.keytab.
Entry for principal oryx/127.0.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.252.keytab.
I20260806 22:40:26.900405 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:34757
--webserver_interface=127.0.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.252.keytab
--principal=oryx/127.0.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:26.989562 11367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:26.989756 11367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:26.989778 11367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:26.990880 11367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:26.990914 11367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:26.990926 11367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:26.990938 11367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:26.990952 11367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:26.992460 11367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.252.keytab
--principal=oryx/127.0.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:34757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:26.992654 11367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:26.992828 11367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:26.997573 11375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:26.997546 11374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.997656 11367 server_base.cc:1034] running on GCE node
W20260806 22:40:26.997555 11381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:26.997979 11367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:26.998267 11367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:26.999406 11367 hybrid_clock.cc:648] HybridClock initialized: now 1786056026999391 us; error 28 us; skew 500 ppm
Aug 06 22:40:26 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.001850 11367 init.cc:377] Logged in from keytab as oryx/127.0.186.252@KRBTEST.COM (short username oryx)
I20260806 22:40:27.002247 11367 webserver.cc:460] Webserver started at http://127.0.186.252:37931/ using document root <none> and password file <none>
I20260806 22:40:27.002455 11367 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.002496 11367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.002588 11367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:27.003615 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/instance:
uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "af2595d25b85a746463c7ede0ab842da"
server_key_iv: "fd99451b90ae9d9d22fd072c1bc2f79a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.003902 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal/instance:
uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "af2595d25b85a746463c7ede0ab842da"
server_key_iv: "fd99451b90ae9d9d22fd072c1bc2f79a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.005077 11367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:27.005790 11392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.005988 11367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:27.006078 11367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "af2595d25b85a746463c7ede0ab842da"
server_key_iv: "fd99451b90ae9d9d22fd072c1bc2f79a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.006138 11367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.040233 11367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.042562 11367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:27.042712 11367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.046617 11367 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.252:34757
I20260806 22:40:27.046664 11466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.252:34757 every 8 connection(s)
I20260806 22:40:27.046995 11367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
I20260806 22:40:27.048306 11467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:27.050382 11467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:27.056254 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 11367
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.252@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:27.060871 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.051240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.0.186.254:46337)
0806 22:40:27.051309 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.051311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.051425 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:40:27.052328 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.052329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.052341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.052402 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.053214 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.053717 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.054792 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.055009 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.055067 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.057817 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.057818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.057819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.057828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.059426 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.059823 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.059824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.059824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.059893 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.060194 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.060194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.060195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.060319 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.060380 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.060709 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.060781 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:40:27.061076 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.051129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44730)
0806 22:40:27.051310 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.051353 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.052216 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.052460 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.052475 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.052756 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.053109 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.053115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.053795 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.053796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.054703 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.054706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.055119 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.055120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.055249 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.055803 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.055812 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.057722 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.059528 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.059534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.059537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.059756 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.059966 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.059967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.059968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.060105 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.060390 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.060395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.060561 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.060664 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.060837 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260806 22:40:27.061604 11467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.252@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
I20260806 22:40:27.065927 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.056892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.0.186.252:34757)
0806 22:40:27.057172 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.057176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.057194 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:40:27.057213 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.057215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.057239 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.057306 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.057910 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.058947 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.059911 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.060392 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.060458 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.062881 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.062888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.062889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.062895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.064730 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.065108 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.065111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.065111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.065166 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.065358 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.065358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.065359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.065529 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.065639 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.065747 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.065828 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":167,"threads_started":1}
I20260806 22:40:27.068339 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.061828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47150)
0806 22:40:27.061894 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.061906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.061956 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.062141 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.062142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.062266 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.062314 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.062316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.063011 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.063012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.063665 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.063668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.064017 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.064018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.064087 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.064410 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.064414 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.065705 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.067327 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.067328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.067328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.067493 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.067707 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.067708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.067708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.067773 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.068093 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.068093 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.068136 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.068191 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.068272 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:40:27.068425 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.061992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.0.186.253:40433)
0806 22:40:27.062036 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.062037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.062044 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:27.062070 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.062070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.062082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.062125 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.062386 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.062912 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.063745 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.063958 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.064012 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.065785 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.065786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.065786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.065793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.067201 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.067567 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.067568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.067568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.067622 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.067891 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.067892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.067892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.068030 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.068096 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.068254 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.068336 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:27.068835 11467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.252@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
I20260806 22:40:27.074680 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.069088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.0.1:46034)
0806 22:40:27.069135 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.069146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.069183 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.069306 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.069306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.069415 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.069448 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.069449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.070015 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.070015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.070473 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.070474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.070782 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.070783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.070816 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.071183 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.071188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.072333 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.073791 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.073792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.073792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.073943 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.074138 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.074138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.074139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.074181 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.074441 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.074442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.074474 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.074518 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.074591 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:40:27.074733 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.069150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.0.186.252:34757)
0806 22:40:27.069202 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.069203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.069212 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:27.069235 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.069235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.069243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.069291 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.069495 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.069935 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.070538 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.070730 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.070780 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.072433 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.072434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.072434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.072440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.073717 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.074024 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.074025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.074026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.074085 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.074257 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.074258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.074258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.074381 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.074440 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.074544 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.074638 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":30}
I20260806 22:40:27.076489 11467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Bootstrap starting.
I20260806 22:40:27.077802 11467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:27.078533 11467 log.cc:824] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:27.079401 11467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: No bootstrap required, opened a new log
I20260806 22:40:27.082417 11467 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } }
I20260806 22:40:27.082561 11467 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:27.082584 11467 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe0b6e8025a413a9f89d58b0b6e491e, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.082681 11467 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } }
I20260806 22:40:27.082991 11467 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:27.082983 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } } }
I20260806 22:40:27.083127 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:40:27.085353 11490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:40:27.085399 11490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:40:27.087831 11367 master_runner.cc:428] Detected that this master fbe0b6e8025a413a9f89d58b0b6e491e is joining an existing cluster
I20260806 22:40:27.087881 11367 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.186.252:34757
I20260806 22:40:27.097538 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.089556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.0.186.254:46337)
0806 22:40:27.089627 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.089629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.090198 (+ 569us) server_negotiation.cc:407] Connection header received
0806 22:40:27.090329 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.090330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.090341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.090372 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.091130 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.091902 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093044 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093340 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093411 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.094949 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.094950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.094951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.094957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.096329 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.096757 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.096758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.096758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.096820 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.097092 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.097092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.097092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.097210 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.097260 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.097410 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.097472 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:40:27.097748 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.089830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44742)
0806 22:40:27.090136 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.090145 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.090248 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.090435 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.090436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.090811 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.091029 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.091032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.092013 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.092015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.092947 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.092950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093808 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.093904 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.094374 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.094379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.094860 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.096410 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.096411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.096412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.096650 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.096893 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.096894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.096894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.097002 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.097269 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.097270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.097317 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.097393 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.097501 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":220,"threads_started":1}
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056025, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.254@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
I20260806 22:40:27.114993 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.089692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.0.186.252:34757)
0806 22:40:27.089739 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.089741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.090100 (+ 359us) server_negotiation.cc:407] Connection header received
0806 22:40:27.090200 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.090201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.090210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.090237 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.090999 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.091722 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.092780 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093060 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093119 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.111890 (+ 18771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.111892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.111892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.111903 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.113451 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114224 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.114225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.114226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.114280 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114452 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.114453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.114453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.114589 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.114635 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.114809 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.114896 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:27.115456 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.089644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.0.1:46040)
0806 22:40:27.090005 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.090017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.090129 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.090295 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.090296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.090769 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.090926 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.090929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.091797 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.091798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.092694 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.092697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093209 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.093313 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.093837 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.093842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.111778 (+ 17936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.113545 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.113546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.113547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.114131 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.114344 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114344 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.114344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.114394 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.114676 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.114677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.114713 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.115171 (+ 458us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.115297 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":278,"threads_started":1}
I20260806 22:40:27.116067 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.100006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.0.1:46056)
0806 22:40:27.100061 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.100073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.100117 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.100558 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.100559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.100735 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.100785 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.100787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.101621 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.101623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.102475 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.102478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.102889 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.102890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.102965 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.103357 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.103363 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.104790 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.114629 (+ 9839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114631 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.114632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.114878 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.115313 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.115314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.115314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.115394 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.115735 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.115736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.115789 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.115836 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.115954 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:40:27.116266 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.090383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47158)
0806 22:40:27.090671 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.090680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.090760 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.090934 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.090935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.091157 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.091416 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.091419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.092312 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.092313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.093368 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.093370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093828 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.093908 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.112092 (+ 18184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.112097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.112834 (+ 737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.114490 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.114491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.114689 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.114886 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.114887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.115022 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.115875 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.115876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.115928 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.115997 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.116106 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":106,"threads_started":1}
I20260806 22:40:27.116523 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.100192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.0.186.252:34757)
0806 22:40:27.100420 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.100422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.100428 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:40:27.100466 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.100466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.100475 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.100529 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.100855 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.101530 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.102576 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.102821 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.102872 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.105018 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.105020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.105020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.105028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.114495 (+ 9467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114967 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.114968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.114968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.115089 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.115475 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.115476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.115477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.115646 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.116180 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.116222 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.116303 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":152,"threads_started":1}
I20260806 22:40:27.116914 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.090233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.0.186.253:40433)
0806 22:40:27.090276 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.090277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.090733 (+ 456us) server_negotiation.cc:407] Connection header received
0806 22:40:27.090828 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.090829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.090840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.090871 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.091510 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.092221 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093463 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.093712 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.093751 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.112954 (+ 19203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.112956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.112956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.112965 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.114407 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.114764 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.114766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.114766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.114825 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.115372 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.115373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.115374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.115527 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.115609 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.116734 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.116821 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:27.117215 10365 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.186.252:34757: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
member_type: NON_VOTER
last_known_addr {
host: "127.0.186.252"
port: 34757
}
attrs {
promote: true
}
}
dest_uuid: "c4ad2cbb63fc4481ad69570692339466"
cas_config_opid_index: 5
I20260806 22:40:27.117532 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 10454
I20260806 22:40:27.117502 10365 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } }
I20260806 22:40:27.121991 11476 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:27.122107 11476 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } }
I20260806 22:40:27.122342 11476 leader_election.cc:288] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337), 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253:40433)
I20260806 22:40:27.122596 11298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "298726ce9e304e24bca54ff4f3c82854" is_pre_election: true
I20260806 22:40:27.122694 11298 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Handling vote request from an unknown peer fbe0b6e8025a413a9f89d58b0b6e491e
I20260806 22:40:27.122728 11298 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate fbe0b6e8025a413a9f89d58b0b6e491e for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 22:40:27.122972 10380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ad2cbb63fc4481ad69570692339466" is_pre_election: true
I20260806 22:40:27.123445 11398 leader_election.cc:302] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbe0b6e8025a413a9f89d58b0b6e491e; no voters: 298726ce9e304e24bca54ff4f3c82854, c4ad2cbb63fc4481ad69570692339466
I20260806 22:40:27.123602 11476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:27.124936 11476 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 22:40:27.124872 11224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.125880 11224 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.186.252:34757
I20260806 22:40:27.125960 11224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.126011 11224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.131253 11432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:40:27.132061 11225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:27.134542 11432 raft_consensus.cc:858] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } }
W20260806 22:40:27.134714 11432 raft_consensus.cc:1565] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } }
I20260806 22:40:27.136775 11432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:27.136942 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.130228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47164)
0806 22:40:27.131940 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.131952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.131998 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.133759 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.133760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.133762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:27.133809 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.133811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.134573 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.134574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.135682 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.135686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.136692 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.136693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.136757 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.136786 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.136836 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.136874 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1689}
I20260806 22:40:27.137223 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.130285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.0.186.253:40433)
0806 22:40:27.130331 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.130333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.133557 (+ 3224us) server_negotiation.cc:407] Connection header received
0806 22:40:27.133593 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.133594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.133621 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.133690 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:27.133864 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.134475 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.135966 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.136521 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.136584 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.136661 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.137113 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.137159 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:27.137428 11298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:40:27.137640 11225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:40:27.138350 11298 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.139420 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ad2cbb63fc4481ad69570692339466. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:27.139504 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:40:27.139541 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:27.139598 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:40:27.139616 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } }
I20260806 22:40:27.139640 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:40:27.140241 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ad2cbb63fc4481ad69570692339466. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.140336 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:40:27.140379 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.140426 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:40:27.140887 11225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.140978 11225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.140975 11432 raft_consensus.cc:1467] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260806 22:40:27.141058 11432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.141039 11225 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: attempting to promote NON_VOTER 298726ce9e304e24bca54ff4f3c82854 to VOTER
I20260806 22:40:27.141300 11225 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } }
I20260806 22:40:27.141790 11298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 LEARNER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:40:27.142009 11224 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:40:27.142661 11224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.142158 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: NON_VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: true } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.142884 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: LEARNER
I20260806 22:40:27.143289 11430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 LEARNER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:40:27.143560 11224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.143641 11224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.143601 11505 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:40:27.143997 11298 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.144032 11430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 LEARNER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.145184 11516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.145274 11516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:27.145608 11224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.145702 11224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.145850 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: NON_VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: true } } }
I20260806 22:40:27.145949 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: LEARNER
I20260806 22:40:27.146273 11225 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: attempting to promote NON_VOTER fbe0b6e8025a413a9f89d58b0b6e491e to VOTER
I20260806 22:40:27.146361 11225 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:27.147514 11298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:40:27.147712 11225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:40:27.147773 11430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 LEARNER]: Refusing update from remote peer c4ad2cbb63fc4481ad69570692339466: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:40:27.147982 11225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:40:27.148247 11225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.148833 11430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.149197 11225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.149268 11225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.149312 11225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.149350 11225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.149381 11225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.149417 11225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.149641 11298 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.150313 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.150442 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:27.150641 11516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.150723 11516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:27.151371 11506 mvcc.cc:205] Tried to move back new op lower bound from 7315685482708787200 to 7315685482570039296. Current Snapshot: MvccSnapshot[applied={T|T < 7315685482580795392}]
I20260806 22:40:27.163640 11367 master.cc:537] Master@127.0.186.252:34757 shutting down...
I20260806 22:40:27.167028 11367 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:27.167181 11367 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:27.167220 11367 tablet_replica.cc:330] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: stopping tablet replica
I20260806 22:40:27.167675 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:35121
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:46337
--webserver_port=46819
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:27.171168 11367 master.cc:552] Master@127.0.186.252:34757 shutdown complete.
I20260806 22:40:27.171221 11367 master_runner.cc:305] Clearing existing system tablet
I20260806 22:40:27.171368 11367 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.171420 11367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.172143 11367 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:27.172765 11527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.172957 11367 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.173018 11367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:27.173043 11367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "af2595d25b85a746463c7ede0ab842da"
server_key_iv: "fd99451b90ae9d9d22fd072c1bc2f79a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.173775 11367 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:27.175185 11367 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:40:27.175256 11367 log.cc:1196] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:40:27.175520 11367 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Deleting consensus metadata
I20260806 22:40:27.175695 11367 master_runner.cc:315] Copying system tablet from 127.0.186.254:46337
I20260806 22:40:27.176311 11367 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.186.254:46337
I20260806 22:40:27.183511 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.176604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:44744)
0806 22:40:27.176736 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.176749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.176810 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.176969 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.176970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.177172 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.177228 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.177230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.178176 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.178177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.178982 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.178987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.179486 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.179488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.179559 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.180060 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.180065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.180478 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.182270 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.182272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.182272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.182495 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.182789 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.182790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.182791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.182864 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.183151 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.183152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.183205 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.183295 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.183408 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260806 22:40:27.183516 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.176783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.0.186.254:46337)
0806 22:40:27.176826 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.176829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.176838 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:27.176869 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.176869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.176881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.176945 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.177308 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.178090 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.179083 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.179405 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.179482 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.180571 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.180573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.180573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.180581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.182178 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.182568 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.182586 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.182586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.182649 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.182958 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.182959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.182959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.183085 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.183143 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.183344 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.183422 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:27.184181 10395 tablet_copy_service.cc:140] P c4ad2cbb63fc4481ad69570692339466: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fbe0b6e8025a413a9f89d58b0b6e491e ({username='oryx', principal='oryx/127.0.186.252@KRBTEST.COM'} at 127.0.0.1:44744)
I20260806 22:40:27.184234 10395 tablet_copy_service.cc:161] P c4ad2cbb63fc4481ad69570692339466: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fbe0b6e8025a413a9f89d58b0b6e491e at {username='oryx', principal='oryx/127.0.186.252@KRBTEST.COM'} at 127.0.0.1:44744: session id = fbe0b6e8025a413a9f89d58b0b6e491e-00000000000000000000000000000000
I20260806 22:40:27.184810 10395 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:40:27.185465 11367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:40:27.186757 11367 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: tablet copy: Starting download of 0 data blocks...
I20260806 22:40:27.186926 11367 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: tablet copy: Starting download of 1 WAL segments...
I20260806 22:40:27.187856 11367 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:40:27.188805 10395 tablet_copy_service.cc:342] P c4ad2cbb63fc4481ad69570692339466: Request end of tablet copy session fbe0b6e8025a413a9f89d58b0b6e491e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.186.252@KRBTEST.COM'} at 127.0.0.1:44744
I20260806 22:40:27.188854 10395 tablet_copy_service.cc:433] P c4ad2cbb63fc4481ad69570692339466: ending tablet copy session fbe0b6e8025a413a9f89d58b0b6e491e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fbe0b6e8025a413a9f89d58b0b6e491e
I20260806 22:40:27.191452 11367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:27.192564 11534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.196192 11537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.196348 11539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:27.199064 11367 server_base.cc:1034] running on GCE node
I20260806 22:40:27.199170 11367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:27.199337 11367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:27.200471 11367 hybrid_clock.cc:648] HybridClock initialized: now 1786056027200477 us; error 38 us; skew 500 ppm
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.201938 11367 init.cc:377] Logged in from keytab as oryx/127.0.186.252@KRBTEST.COM (short username oryx)
I20260806 22:40:27.202196 11367 webserver.cc:460] Webserver started at http://127.0.186.252:35587/ using document root <none> and password file <none>
I20260806 22:40:27.202306 11367 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.202339 11367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.203070 11367 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:27.203717 11551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.203832 11367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.203874 11367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "af2595d25b85a746463c7ede0ab842da"
server_key_iv: "fd99451b90ae9d9d22fd072c1bc2f79a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.203943 11367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.220021 11367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.221817 11367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.225468 11367 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.252:34757
I20260806 22:40:27.225931 11367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
I20260806 22:40:27.226167 11629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.252:34757 every 8 connection(s)
I20260806 22:40:27.226828 11630 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:27.227335 11630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Bootstrap starting.
I20260806 22:40:27.229590 11630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:27.229791 11630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: Bootstrap complete.
I20260806 22:40:27.229975 11630 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:27.230065 11630 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe0b6e8025a413a9f89d58b0b6e491e, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.230159 11630 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:27.230345 11630 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:27.230350 11636 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.230562 11636 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:27.231802 11651 catalog_manager.cc:1259] Loaded cluster ID: 37389e40cc1145b1a630b0c6ba518290
I20260806 22:40:27.232298 11651 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: loading cluster ID for follower catalog manager: success
I20260806 22:40:27.233428 11651 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: acquiring CA information for follower catalog manager: success
I20260806 22:40:27.233788 11651 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:40:27.266844 11524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:27.267078 11524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:27.267117 11524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:27.268674 11524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:27.268718 11524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:27.268743 11524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:27.270352 11524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:35121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=46819
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:27.270592 11524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:27.270816 11524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:27.276561 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.276610 11662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.276651 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:27.276821 11524 server_base.cc:1034] running on GCE node
I20260806 22:40:27.277041 11524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:27.277289 11524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:27.278431 11524 hybrid_clock.cc:648] HybridClock initialized: now 1786056027278417 us; error 29 us; skew 500 ppm
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.280732 11524 init.cc:377] Logged in from keytab as oryx/127.0.186.193@KRBTEST.COM (short username oryx)
I20260806 22:40:27.281147 11524 webserver.cc:460] Webserver started at http://127.0.186.193:46819/ using document root <none> and password file <none>
I20260806 22:40:27.281358 11524 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.281422 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.282681 11524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:27.283185 11673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.283344 11524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:27.283424 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "308bc0d3695040e6a296bbb4613fea1d"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "a89971886170658a9ace4f35c813dea7"
server_key_iv: "dcd732129bb2a98ff2de480dbad54a1a"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.283831 11524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.302119 11524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.304385 11524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:27.304523 11524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.304739 11524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:27.305035 11524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:27.305069 11524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.305097 11524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:27.305121 11524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.318017 11524 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:35121
I20260806 22:40:27.318230 11843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:35121 every 8 connection(s)
I20260806 22:40:27.318455 11524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:27.324590 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 11524
I20260806 22:40:27.324784 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 10669
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
I20260806 22:40:27.342782 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54069 (local address 127.0.186.253:40433)
0806 22:40:27.320009 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.320011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.320300 (+ 289us) server_negotiation.cc:407] Connection header received
0806 22:40:27.320719 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.320720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.320729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.320771 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.321811 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322609 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324248 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.324530 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324580 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.338359 (+ 13779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.338361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.338361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.338368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.341198 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.341780 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.341782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.341782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.341842 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.342144 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.342145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.342145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.342270 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.342328 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.342607 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.342692 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:27.343441 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.186.193:54069)
0806 22:40:27.320232 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.320243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.320494 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.320789 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.320790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.321284 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.321717 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.321723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322690 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.322691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.324164 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.324167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.324628 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.324805 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.336200 (+ 11395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.336211 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.338237 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.341365 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.341374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.341377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.341698 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.341938 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.341939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.341939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.342080 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.342328 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.342336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.342486 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.342578 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.342750 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":32,"thread_start_us":200,"threads_started":1}
I20260806 22:40:27.343954 11846 heartbeater.cc:347] Connected to a master server at 127.0.186.253:40433
I20260806 22:40:27.344065 11846 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.344275 11846 heartbeater.cc:511] Master 127.0.186.253:40433 requested a full tablet report, sending...
I20260806 22:40:27.344826 11286 ts_manager.cc:194] Registered new tserver with Master: 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
I20260806 22:40:27.345901 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:49457 (local address 127.0.186.252:34757)
0806 22:40:27.319478 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.319480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.319713 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:40:27.320828 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.320830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.320843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.320918 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.321845 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322819 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324176 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.324481 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324544 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.341663 (+ 17119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.341666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.341666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.341674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.344599 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.345054 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.345056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.345056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.345122 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.345315 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.345316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.345316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.345457 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.345517 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.345704 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.345801 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260806 22:40:27.346194 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.186.193:49457)
0806 22:40:27.319540 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.319584 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.320464 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.320932 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.320933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.321381 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.321719 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.321726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322889 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.322890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.324075 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.324078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.324613 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.338868 (+ 14253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.340075 (+ 1207us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.340080 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.341547 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.344682 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.344683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.344684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.344965 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.345185 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.345186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.345186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.345243 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.345546 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.345548 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.345589 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.345975 (+ 386us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.346087 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":15,"thread_start_us":88,"threads_started":1}
I20260806 22:40:27.346864 11844 heartbeater.cc:347] Connected to a master server at 127.0.186.252:34757
I20260806 22:40:27.346911 11844 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.346978 11844 heartbeater.cc:511] Master 127.0.186.252:34757 requested a full tablet report, sending...
I20260806 22:40:27.347324 11573 ts_manager.cc:194] Registered new tserver with Master: 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:27.357311 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:40993
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:46337
--webserver_port=36259
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:27.358480 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:58087 (local address 127.0.186.254:46337)
0806 22:40:27.319722 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.319724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.319914 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:40:27.320577 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.320579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.320593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.320640 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.321834 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322344 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324399 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.324676 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.324740 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.356065 (+ 31325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.356067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.356067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.356074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.357433 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.357761 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.357764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.357764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.357832 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.358025 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.358026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.358026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.358154 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.358218 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.358315 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.358396 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:27.358584 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.319583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:58087)
0806 22:40:27.319849 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.319859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.320465 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.320724 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.320729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.321366 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.321719 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.321726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.322545 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.322546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.324326 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.324328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.353828 (+ 29500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.353830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.353977 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.354418 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.354423 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.355952 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.357526 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.357528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.357528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.357685 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.357929 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.357930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.357930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.357976 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.358210 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.358210 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.358245 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.358291 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.358373 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":54,"thread_start_us":197,"threads_started":1}
I20260806 22:40:27.358971 11845 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:27.359036 11845 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.359109 11845 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:27.359442 10365 ts_manager.cc:194] Re-registered known tserver with Master: 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
I20260806 22:40:27.359817 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:58087
W20260806 22:40:27.439361 11853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:27.439561 11853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:27.439594 11853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:27.440711 11853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:27.440750 11853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:27.440785 11853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:27.442286 11853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:40993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=36259
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:27.442610 11853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:27.442832 11853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:27.447916 11863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.447958 11860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.447892 11861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:27.448252 11853 server_base.cc:1034] running on GCE node
I20260806 22:40:27.448414 11853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:27.448670 11853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:27.449820 11853 hybrid_clock.cc:648] HybridClock initialized: now 1786056027449806 us; error 30 us; skew 500 ppm
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.452380 11853 init.cc:377] Logged in from keytab as oryx/127.0.186.194@KRBTEST.COM (short username oryx)
I20260806 22:40:27.452896 11853 webserver.cc:460] Webserver started at http://127.0.186.194:36259/ using document root <none> and password file <none>
I20260806 22:40:27.453123 11853 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.453186 11853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.454525 11853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:27.455063 11874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.455228 11853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:27.455335 11853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "91f860820c864aef8989d22e2e64aa62"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "ae63ef3ce5890e6616a10ea3606d223a"
server_key_iv: "c073d15161fb9d89af478770a836c9b4"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.455672 11853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.471201 11853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.473441 11853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:27.473583 11853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.473798 11853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:27.474112 11853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:27.474146 11853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.474169 11853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:27.474190 11853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.482055 11853 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:40993
I20260806 22:40:27.482127 12037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:40993 every 8 connection(s)
I20260806 22:40:27.482450 11853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:27.483039 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 11853
I20260806 22:40:27.483210 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 10892
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:27.507508 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.489652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:51267 (local address 127.0.186.254:46337)
0806 22:40:27.489706 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.489709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.496405 (+ 6696us) server_negotiation.cc:407] Connection header received
0806 22:40:27.496521 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.496522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.496535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.496562 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.497810 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.498556 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.500087 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.500432 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.500507 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.504193 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.504195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.504196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.504203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.505774 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.506369 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.506371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.506371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.506449 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.506749 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.506751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.506751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.506911 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.506979 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.507331 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.507416 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
I20260806 22:40:27.507719 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:44893
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:46337
--webserver_port=37791
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:27.507763 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.483514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:51267)
0806 22:40:27.496319 (+ 12805us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.496332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.496437 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.496733 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.496738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.497370 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.497689 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.497696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.498679 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.498680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.499982 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.499985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.500579 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.500580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.500813 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.501966 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.501976 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.504077 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.505865 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.505872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.505875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.506290 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.506515 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.506516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.506516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.506656 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.506987 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.506994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.507156 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.507284 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.507455 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12785,"thread_start_us":58,"threads_started":1}
I20260806 22:40:27.509706 12040 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:27.509816 12040 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.510013 12040 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:27.510406 10365 ts_manager.cc:194] Re-registered known tserver with Master: 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
I20260806 22:40:27.510881 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:51267
I20260806 22:40:27.511637 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.487739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:41363 (local address 127.0.186.253:40433)
0806 22:40:27.500402 (+ 12663us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.500405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.500414 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:27.500449 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.500450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.500462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.500513 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.503679 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.504315 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.505465 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.505729 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.505770 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.508316 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.508318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.508318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.508326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.509775 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.510655 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.510657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.510657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.510727 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.511052 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.511053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.511054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.511198 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.511253 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.511468 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.511554 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12627}
I20260806 22:40:27.512010 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.483209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.186.194:41363)
0806 22:40:27.491393 (+ 8184us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.491447 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.492276 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.503051 (+ 10775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.503052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.503385 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.503529 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.503532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.504655 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.504656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.505352 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.505355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.505852 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.505853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.505921 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.506457 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.506461 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.508231 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.510262 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.510263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.510263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.510548 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.510800 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.510801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.510801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.510897 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.511306 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.511307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.511345 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.511840 (+ 495us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.511937 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8129,"thread_start_us":100,"threads_started":1}
I20260806 22:40:27.512329 12043 heartbeater.cc:347] Connected to a master server at 127.0.186.253:40433
I20260806 22:40:27.512382 12043 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.512449 12043 heartbeater.cc:511] Master 127.0.186.253:40433 requested a full tablet report, sending...
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
I20260806 22:40:27.512717 11286 ts_manager.cc:194] Registered new tserver with Master: 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
I20260806 22:40:27.515089 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.492121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:34177 (local address 127.0.186.252:34757)
0806 22:40:27.492175 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.492177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.507938 (+ 15761us) server_negotiation.cc:407] Connection header received
0806 22:40:27.508041 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.508042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.508053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.508083 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.508523 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.509132 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.510495 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.510810 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.510847 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.512942 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.512943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.512944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.512951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.514124 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.514463 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.514464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.514464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.514508 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.514658 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.514659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.514659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.514747 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.514789 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.514963 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.515029 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:40:27.515108 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.490636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.186.194:34177)
0806 22:40:27.507865 (+ 17229us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.507877 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.507969 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.508142 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.508142 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.508280 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.508437 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.508439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.509205 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.509833 (+ 628us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.510400 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.510403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.510934 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.510935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.511002 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.511537 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.511542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.512834 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.514198 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.514199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.514199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.514397 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.514548 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.514549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.514549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.514609 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.514798 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.514799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.514840 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.514924 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.515021 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17217,"thread_start_us":604,"threads_started":1}
I20260806 22:40:27.516026 12039 heartbeater.cc:347] Connected to a master server at 127.0.186.252:34757
I20260806 22:40:27.516073 12039 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.516112 12039 heartbeater.cc:511] Master 127.0.186.252:34757 requested a full tablet report, sending...
I20260806 22:40:27.516383 11573 ts_manager.cc:194] Registered new tserver with Master: 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
W20260806 22:40:27.597774 12049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:27.597983 12049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:27.598016 12049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:27.599159 12049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:27.599198 12049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:27.599232 12049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:27.600742 12049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:44893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=37791
--tserver_master_addrs=127.0.186.254:46337,127.0.186.253:40433,127.0.186.252:34757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:27.600984 12049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:27.601184 12049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:27.607051 12057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.607089 12059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.607239 12056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:27.607412 12049 server_base.cc:1034] running on GCE node
I20260806 22:40:27.607583 12049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:27.607784 12049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:27.608914 12049 hybrid_clock.cc:648] HybridClock initialized: now 1786056027608906 us; error 28 us; skew 500 ppm
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.611961 12049 init.cc:377] Logged in from keytab as oryx/127.0.186.195@KRBTEST.COM (short username oryx)
I20260806 22:40:27.612388 12049 webserver.cc:460] Webserver started at http://127.0.186.195:37791/ using document root <none> and password file <none>
I20260806 22:40:27.612600 12049 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.612668 12049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.614004 12049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:27.614531 12069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.614691 12049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:27.614774 12049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "8633da54eb634804b8ed726e4eee00df"
format_stamp: "Formatted at 2026-08-06 22:40:26 on dist-test-slave-gjrz"
server_key: "163aa033dfb4a331d781495681466195"
server_key_iv: "77659545379af786b49839b214c66e92"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.615099 12049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.635176 12049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.637022 12049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:27.637184 12049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.637390 12049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:27.637720 12049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:27.637758 12049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.637786 12049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:27.637801 12049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.646028 12049 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:44893
I20260806 22:40:27.646082 12233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:44893 every 8 connection(s)
I20260806 22:40:27.646410 12049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.252@KRBTEST.COM
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.253@KRBTEST.COM
Aug 06 22:40:27 dist-test-slave-gjrz krb5kdc[10298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:27.654975 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 12049
I20260806 22:40:27.657371 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:33161 (local address 127.0.186.252:34757)
0806 22:40:27.647361 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.647363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.647664 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:40:27.648425 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.648426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.648438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.648469 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.649350 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.649838 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651160 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651473 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651535 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.654125 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.654127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.654127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.654136 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.655777 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656233 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.656234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.656235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.656297 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656622 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.656624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.656624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.656765 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.656809 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.657200 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.657282 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:27.657549 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:42125 (local address 127.0.186.253:40433)
0806 22:40:27.647617 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.647619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.647671 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:40:27.648424 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.648425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.648438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.648469 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.649357 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.650114 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651196 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651508 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651560 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.654483 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.654484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.654485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.654492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.656371 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656721 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.656723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.656723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.656783 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656996 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.656997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.656997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.657141 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.657183 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.657385 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.657462 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:27.657856 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.186.195:33161)
0806 22:40:27.647503 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.647552 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.648301 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.648555 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.648561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.649028 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.649239 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.649245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.649935 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.649936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651052 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.651055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651593 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651715 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.652295 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.652306 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.654031 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.655871 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.655879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.655881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.656134 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.656376 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.656377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.656526 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.656858 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.656865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.657016 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.657467 (+ 451us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.657641 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":26,"thread_start_us":148,"threads_started":1}
I20260806 22:40:27.658231 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.186.195:42125)
0806 22:40:27.647604 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.647622 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.648308 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.648561 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.648561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.649039 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.649239 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.649245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.650194 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.650195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651081 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.651083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651607 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651715 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.652322 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.652326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.654391 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.656438 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.656440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.656635 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.656860 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.656861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.656924 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.657231 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.657232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.657271 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.657665 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.657771 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":10,"thread_start_us":63,"threads_started":1}
I20260806 22:40:27.658324 12235 heartbeater.cc:347] Connected to a master server at 127.0.186.252:34757
I20260806 22:40:27.658408 12235 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.658545 12237 heartbeater.cc:347] Connected to a master server at 127.0.186.253:40433
I20260806 22:40:27.658567 12243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:51077)
0806 22:40:27.647642 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.647652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.648308 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.648825 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.648826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.649029 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:27.649238 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.649243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.650194 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.650195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651206 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.651208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651570 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.651713 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.652401 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:27.652405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:27.654941 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:27.656970 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.656970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.656971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.657162 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.657358 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:27.657359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:27.657359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:27.657412 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:27.658279 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:27.658279 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:27.658321 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.658383 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.658475 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":7,"thread_start_us":95,"threads_started":1}
I20260806 22:40:27.658588 12237 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.658591 12235 heartbeater.cc:511] Master 127.0.186.252:34757 requested a full tablet report, sending...
I20260806 22:40:27.658667 12237 heartbeater.cc:511] Master 127.0.186.253:40433 requested a full tablet report, sending...
I20260806 22:40:27.659083 11573 ts_manager.cc:194] Registered new tserver with Master: 8633da54eb634804b8ed726e4eee00df (127.0.186.195:44893)
I20260806 22:40:27.659106 11286 ts_manager.cc:194] Registered new tserver with Master: 8633da54eb634804b8ed726e4eee00df (127.0.186.195:44893)
I20260806 22:40:27.659183 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.647112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:51077 (local address 127.0.186.254:46337)
0806 22:40:27.647167 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.647170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.647702 (+ 532us) server_negotiation.cc:407] Connection header received
0806 22:40:27.648400 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.648401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.648413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.648472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.649327 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.650096 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651305 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.651490 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.651543 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.655328 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.655330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.655330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.655337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.656897 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.657235 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.657236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.657237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.657294 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.658055 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.658057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.658057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.658195 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.658253 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.659018 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.659094 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:27.659520 12236 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46337
I20260806 22:40:27.659576 12236 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:27.659627 12236 heartbeater.cc:511] Master 127.0.186.254:46337 requested a full tablet report, sending...
I20260806 22:40:27.659849 10365 ts_manager.cc:194] Re-registered known tserver with Master: 8633da54eb634804b8ed726e4eee00df (127.0.186.195:44893)
I20260806 22:40:27.660233 10365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:51077
I20260806 22:40:27.665277 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.656598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.0.186.252:34757)
0806 22:40:27.656928 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.656930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.657090 (+ 160us) server_negotiation.cc:407] Connection header received
0806 22:40:27.657110 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.657111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.657122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.657173 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.658025 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.659790 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.660761 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.661041 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.661092 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.662573 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.662574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.662575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.662582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.664139 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.664503 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.664504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.664505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.664577 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.664757 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.664757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.664758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.664862 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.664949 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.665045 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.665127 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":52,"threads_started":1}
I20260806 22:40:27.666596 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.660032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.0.186.254:46337)
0806 22:40:27.660068 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.660069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.660461 (+ 392us) server_negotiation.cc:407] Connection header received
0806 22:40:27.660549 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.660550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.660559 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.660584 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.661297 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.661861 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.662523 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.662782 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.662850 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.664020 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.664021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.664022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.664029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.665583 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.665919 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.665920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.665920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.665978 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.666163 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.666164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.666164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.666284 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.666385 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.666447 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.666512 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:27.667804 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.660534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.0.186.253:40433)
0806 22:40:27.660577 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.660579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.660816 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:40:27.660894 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.660895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.660903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.660928 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.661883 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.662588 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.663519 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.663832 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.663870 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.665170 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.665171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.665171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:27.665178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.666730 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.667075 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.667075 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.667076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.667129 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:27.667337 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:27.667338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:27.667338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:27.667456 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.667560 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.667661 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.667732 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:40:27.668087 10365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:27.669682 10365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:27.673898 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.670404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.0.1:46082)
0806 22:40:27.670500 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.670508 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.670547 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.671084 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.671085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.671088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:27.671132 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.671134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.671908 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.671909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.672876 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.672880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.673658 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.673659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.673725 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.673729 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.673788 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.673831 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260806 22:40:27.673906 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.670441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.0.186.252:34757)
0806 22:40:27.670975 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.670977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.670984 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:27.671003 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.671004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.671021 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.671072 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:27.671198 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.671844 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.672943 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.673468 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.673536 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.673620 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.673798 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.673848 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515}
I20260806 22:40:27.674535 11636 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ad2cbb63fc4481ad69570692339466. Latest consensus state: current_term: 1 leader_uuid: "c4ad2cbb63fc4481ad69570692339466" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:27.674631 11636 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:27.677350 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35121 (local address 127.0.0.1:39346)
0806 22:40:27.674076 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.674086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.674141 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.674364 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.674365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.674367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:27.674405 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.674406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.675677 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.675678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.676370 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.676372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.677125 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.677127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.677188 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.677193 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.677240 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.677281 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":107,"threads_started":1}
I20260806 22:40:27.677640 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.0.186.193:35121)
0806 22:40:27.674115 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.674120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.674131 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:27.674209 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.674212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.674228 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.674300 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:27.674476 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.675594 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.676449 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.677043 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.677105 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.677174 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.677370 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.677417 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":183,"threads_started":1}
I20260806 22:40:27.678308 11752 tablet_service.cc:1467] Processing CreateTablet for tablet 1865c126c935406b99feff80e789485d (DEFAULT_TABLE table=test-table [id=0aaadecb57fb45a8b299f3303881f2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:27.678747 11752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865c126c935406b99feff80e789485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:27.679091 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.0.186.194:40993)
0806 22:40:27.674265 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.674268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.674903 (+ 635us) server_negotiation.cc:407] Connection header received
0806 22:40:27.675367 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.675371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.675388 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.675467 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:27.675577 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.676397 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.677981 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.678733 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.678875 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.678939 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.678966 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.679008 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":84,"threads_started":1}
I20260806 22:40:27.679337 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:40993 (local address 127.0.0.1:59756)
0806 22:40:27.674836 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.674847 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.674968 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.675465 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.675466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.675467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:27.675511 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.675513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.676480 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.676481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.677884 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.677887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.678817 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.678818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.678877 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.678882 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.678924 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.678961 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"thread_start_us":57,"threads_started":1}
I20260806 22:40:27.680426 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44893 (local address 127.0.0.1:43688)
0806 22:40:27.673966 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:27.673977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:27.674018 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:27.677240 (+ 3222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:27.677242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:27.677243 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:27.677279 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.677281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.678605 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.678607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.679551 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:27.679554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.680217 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.680218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:27.680277 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.680282 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:27.680326 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:40:27.680366 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":96,"threads_started":1}
I20260806 22:40:27.680501 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.673601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.0.186.195:44893)
0806 22:40:27.677027 (+ 3426us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.677032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.677042 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:27.677116 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.677118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.677133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.677237 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:27.677335 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.678489 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.679623 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.680140 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.680203 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.680324 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.680350 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.680392 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3182,"thread_start_us":71,"threads_started":1}
I20260806 22:40:27.681008 12140 tablet_service.cc:1467] Processing CreateTablet for tablet 1865c126c935406b99feff80e789485d (DEFAULT_TABLE table=test-table [id=0aaadecb57fb45a8b299f3303881f2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:27.681298 12140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865c126c935406b99feff80e789485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:27.681716 12279 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Bootstrap starting.
I20260806 22:40:27.682228 11947 tablet_service.cc:1467] Processing CreateTablet for tablet 1865c126c935406b99feff80e789485d (DEFAULT_TABLE table=test-table [id=0aaadecb57fb45a8b299f3303881f2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:27.682405 12279 tablet_bootstrap.cc:654] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:27.682546 11947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865c126c935406b99feff80e789485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:27.682838 12279 log.cc:824] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:27.683917 12279 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: No bootstrap required, opened a new log
I20260806 22:40:27.684046 12279 ts_tablet_manager.cc:1397] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:27.684578 12281 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Bootstrap starting.
I20260806 22:40:27.685395 12281 tablet_bootstrap.cc:654] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:27.685762 12281 log.cc:824] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:27.686264 12284 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Bootstrap starting.
I20260806 22:40:27.686599 12281 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: No bootstrap required, opened a new log
I20260806 22:40:27.686662 12281 ts_tablet_manager.cc:1397] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:27.686810 12284 tablet_bootstrap.cc:654] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:27.687062 12284 log.cc:824] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:27.687708 12284 tablet_bootstrap.cc:492] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: No bootstrap required, opened a new log
I20260806 22:40:27.687773 12284 ts_tablet_manager.cc:1397] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:27.688014 12279 raft_consensus.cc:348] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.688196 12279 raft_consensus.cc:374] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:27.688241 12279 raft_consensus.cc:729] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308bc0d3695040e6a296bbb4613fea1d, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.688351 12279 consensus_queue.cc:260] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.688683 12279 ts_tablet_manager.cc:1428] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260806 22:40:27.688776 11845 heartbeater.cc:503] Master 127.0.186.254:46337 was elected leader, sending a full tablet report...
I20260806 22:40:27.689111 12281 raft_consensus.cc:348] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.689257 12281 raft_consensus.cc:374] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:27.689291 12281 raft_consensus.cc:729] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8633da54eb634804b8ed726e4eee00df, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.689571 12284 raft_consensus.cc:348] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.689723 12284 raft_consensus.cc:374] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:27.689384 12281 consensus_queue.cc:260] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.689757 12284 raft_consensus.cc:729] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f860820c864aef8989d22e2e64aa62, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.689874 12284 consensus_queue.cc:260] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:27.689939 12281 ts_tablet_manager.cc:1428] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:27.690130 12284 ts_tablet_manager.cc:1428] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:40:27.690236 12040 heartbeater.cc:503] Master 127.0.186.254:46337 was elected leader, sending a full tablet report...
I20260806 22:40:27.690423 12236 heartbeater.cc:503] Master 127.0.186.254:46337 was elected leader, sending a full tablet report...
W20260806 22:40:27.733194 12045 tablet.cc:2369] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:27.819592 11847 tablet.cc:2369] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:27.897256 12240 tablet.cc:2369] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:28.029642 12288 raft_consensus.cc:482] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:28.029779 12288 raft_consensus.cc:504] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:28.030220 12288 leader_election.cc:288] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993), 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
I20260806 22:40:28.034004 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.030501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:49893 (local address 127.0.186.194:40993)
0806 22:40:28.030559 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.030561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.030572 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:28.030600 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.030601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.030611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.030670 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:28.030816 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.031613 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033068 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.033627 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033699 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.033708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.033893 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.033934 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:28.034005 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.030515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:38549 (local address 127.0.186.193:35121)
0806 22:40:28.030567 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.030569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.030578 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:28.030652 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.030653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.030662 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.030706 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:28.030815 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.031610 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033086 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.033622 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.033708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.033893 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.033933 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:40:28.034020 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.030446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35121 (local address 127.0.186.195:38549)
0806 22:40:28.030533 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:28.030541 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:28.030570 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:28.030714 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:28.030715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:28.030716 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:28.030758 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:28.030759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.031692 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.031694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:28.032976 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:28.032979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.033718 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:28.033815 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.033822 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:40:28.033880 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:28.033926 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":128}
I20260806 22:40:28.034030 12243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.030437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:40993 (local address 127.0.186.195:49893)
0806 22:40:28.030476 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:28.030487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:28.030528 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:28.030673 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:28.030674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:28.030676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:28.030751 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:28.030753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.031702 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.031703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:28.032976 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:28.032979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.033718 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.033720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:28.033815 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.033822 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:40:28.033880 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:28.033926 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":37}
I20260806 22:40:28.034374 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865c126c935406b99feff80e789485d" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308bc0d3695040e6a296bbb4613fea1d" is_pre_election: true
I20260806 22:40:28.034458 11972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865c126c935406b99feff80e789485d" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f860820c864aef8989d22e2e64aa62" is_pre_election: true
I20260806 22:40:28.034550 11781 raft_consensus.cc:2454] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 0.
I20260806 22:40:28.034587 11972 raft_consensus.cc:2454] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 0.
I20260806 22:40:28.034793 12073 leader_election.cc:302] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 308bc0d3695040e6a296bbb4613fea1d, 8633da54eb634804b8ed726e4eee00df, 91f860820c864aef8989d22e2e64aa62; no voters:
I20260806 22:40:28.034935 12288 raft_consensus.cc:2781] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:28.035012 12288 raft_consensus.cc:482] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:28.035035 12288 raft_consensus.cc:3037] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:28.035826 12288 raft_consensus.cc:504] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:28.035957 12288 leader_election.cc:288] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 election: Requested vote from peers 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993), 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121)
I20260806 22:40:28.036113 11972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865c126c935406b99feff80e789485d" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f860820c864aef8989d22e2e64aa62"
I20260806 22:40:28.036111 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865c126c935406b99feff80e789485d" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308bc0d3695040e6a296bbb4613fea1d"
I20260806 22:40:28.036182 11972 raft_consensus.cc:3037] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:28.036185 11781 raft_consensus.cc:3037] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:28.036839 11781 raft_consensus.cc:2454] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 1.
I20260806 22:40:28.036846 11972 raft_consensus.cc:2454] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 1.
I20260806 22:40:28.036988 12073 leader_election.cc:302] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308bc0d3695040e6a296bbb4613fea1d, 8633da54eb634804b8ed726e4eee00df; no voters:
I20260806 22:40:28.037098 12288 raft_consensus.cc:2781] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:28.037253 12288 raft_consensus.cc:686] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 1 LEADER]: Becoming Leader. State: Replica: 8633da54eb634804b8ed726e4eee00df, State: Running, Role: LEADER
I20260806 22:40:28.037345 12288 consensus_queue.cc:237] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } }
I20260806 22:40:28.038337 10365 catalog_manager.cc:5495] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df reported cstate change: term changed from 0 to 1, leader changed from <none> to 8633da54eb634804b8ed726e4eee00df (127.0.186.195). New cstate: current_term: 1 leader_uuid: "8633da54eb634804b8ed726e4eee00df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:28.116284 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.113921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.0.186.195:44893)
0806 22:40:28.113982 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.113984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.113992 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:28.114036 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.114037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.114057 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.114119 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:28.114261 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.114786 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.115573 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.115760 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.115816 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.115978 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:28.116082 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:28.116154 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.116186 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.116207 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:40:28.118218 11972 raft_consensus.cc:1261] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Refusing update from remote peer 8633da54eb634804b8ed726e4eee00df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:28.118235 11781 raft_consensus.cc:1261] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Refusing update from remote peer 8633da54eb634804b8ed726e4eee00df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:28.118553 12288 consensus_queue.cc:1035] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:28.118721 12295 consensus_queue.cc:1035] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [LEADER]: Connected to new peer: Peer: permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:28.123672 10363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:28.123772 10363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:28.124130 12304 mvcc.cc:205] Tried to move back new op lower bound from 7315685491169198080 to 7315685490841493504. Current Snapshot: MvccSnapshot[applied={T|T < 7315685491169198080 or (T in {7315685491169198080})}]
I20260806 22:40:28.125344 10363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4ad2cbb63fc4481ad69570692339466: Sending DeleteTablet for 3 replicas of tablet 1865c126c935406b99feff80e789485d
I20260806 22:40:28.125660 11947 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865c126c935406b99feff80e789485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:28 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:59756
I20260806 22:40:28.125663 12140 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865c126c935406b99feff80e789485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:28 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:43688
I20260806 22:40:28.125865 12314 tablet_replica.cc:330] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: stopping tablet replica
I20260806 22:40:28.125890 11752 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865c126c935406b99feff80e789485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:28 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:39346
I20260806 22:40:28.126063 12314 raft_consensus.cc:2229] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:28.126104 12315 tablet_replica.cc:330] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: stopping tablet replica
I20260806 22:40:28.126210 12315 raft_consensus.cc:2229] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:28.126292 12314 raft_consensus.cc:2258] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:28.126305 12315 raft_consensus.cc:2258] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:28.126436 12316 tablet_replica.cc:330] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: stopping tablet replica
I20260806 22:40:28.126531 12316 raft_consensus.cc:2229] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:28.126675 12316 raft_consensus.cc:2258] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:28.127017 12314 ts_tablet_manager.cc:1905] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:28.127027 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 10316
I20260806 22:40:28.135460 12316 ts_tablet_manager.cc:1905] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:28.135557 12315 ts_tablet_manager.cc:1905] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:28.137214 12314 ts_tablet_manager.cc:1918] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:28.137301 12315 ts_tablet_manager.cc:1918] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:28.137382 12315 log.cc:1196] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/1865c126c935406b99feff80e789485d
I20260806 22:40:28.137511 12314 log.cc:1196] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/1865c126c935406b99feff80e789485d
I20260806 22:40:28.137630 12315 ts_tablet_manager.cc:1939] T 1865c126c935406b99feff80e789485d P 8633da54eb634804b8ed726e4eee00df: Deleting consensus metadata
I20260806 22:40:28.137907 12314 ts_tablet_manager.cc:1939] T 1865c126c935406b99feff80e789485d P 91f860820c864aef8989d22e2e64aa62: Deleting consensus metadata
W20260806 22:40:28.142004 11875 connection.cc:535] server connection from 127.0.0.1:59756 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:40:28.142329 12073 connection.cc:535] server connection from 127.0.0.1:43688 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:40:28.147400 12316 ts_tablet_manager.cc:1918] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:28.147490 12316 log.cc:1196] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/1865c126c935406b99feff80e789485d
I20260806 22:40:28.147760 12316 ts_tablet_manager.cc:1939] T 1865c126c935406b99feff80e789485d P 308bc0d3695040e6a296bbb4613fea1d: Deleting consensus metadata
I20260806 22:40:28.147779 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.126328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.0.186.252:34757)
0806 22:40:28.126380 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.126382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.139622 (+ 13240us) server_negotiation.cc:407] Connection header received
0806 22:40:28.139735 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.139736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.139748 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.139778 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:28.140628 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.141393 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.142637 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.142928 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.142975 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.144842 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:28.144844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:28.144844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:28.144851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:28.146442 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.146726 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.146727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.146727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.146786 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.147005 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.147006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.147007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.147176 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:28.147564 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.147612 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.147691 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 22:40:28.148161 11677 connection.cc:441] server connection from 127.0.0.1:39346 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39346 (request call id 1) could send its response
I20260806 22:40:28.150767 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.143936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.0.186.253:40433)
0806 22:40:28.143993 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.143995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.144071 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:40:28.144129 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.144130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.144140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.144186 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:28.144747 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.145471 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.146409 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.146728 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.146806 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.148423 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:28.148424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:28.148424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:28.148433 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:28.149835 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.150120 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.150121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.150122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.150185 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.150317 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.150318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.150318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.150443 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:28.150534 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.150595 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.150674 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:28.156760 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:28.143734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.0.186.252:34757)
0806 22:40:28.143777 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:28.143779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:28.150754 (+ 6975us) server_negotiation.cc:407] Connection header received
0806 22:40:28.150830 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:28.150831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:28.150841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:28.150891 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:28.151577 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.152030 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.153016 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:28.153192 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:28.153242 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:28.154451 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:28.154453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:28.154453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:28.154460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:28.155798 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.156135 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.156136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.156136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.156180 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:28.156364 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:28.156365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:28.156365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:28.156454 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:28.156522 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:28.156649 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:40:28.156702 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:29.123746 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.123379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:53845)
0806 22:40:29.123529 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.123653 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
W20260806 22:40:29.124102 12236 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:46337 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.124601 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.124370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:36547)
0806 22:40:29.124476 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.124545 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":49,"threads_started":1}
I20260806 22:40:29.124601 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.124362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:42541)
0806 22:40:29.124477 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.124553 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
W20260806 22:40:29.124781 11845 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:46337 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
W20260806 22:40:29.124863 12040 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:46337 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.625339 12333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4ad2cbb63fc4481ad69570692339466)
I20260806 22:40:29.625483 12333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:29.625730 12333 leader_election.cc:288] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337), fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252:34757)
I20260806 22:40:29.626212 12344 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c4ad2cbb63fc4481ad69570692339466)
I20260806 22:40:29.626132 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.625892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45220)
0806 22:40:29.626005 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.626061 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":43,"threads_started":1}
I20260806 22:40:29.626359 12344 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
W20260806 22:40:29.626533 11265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.626933 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.626843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45236)
0806 22:40:29.626883 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.626906 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:40:29.627051 11265 leader_election.cc:334] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337): Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.627213 12344 leader_election.cc:288] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337), 298726ce9e304e24bca54ff4f3c82854 (127.0.186.253:40433)
I20260806 22:40:29.627339 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.626953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45244)
0806 22:40:29.627161 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.627214 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":51,"threads_started":1}
W20260806 22:40:29.627460 11556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.628062 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.627910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45248)
0806 22:40:29.627958 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.628035 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:40:29.628173 11556 leader_election.cc:334] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337): Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.629247 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.626059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.0.186.252:34757)
0806 22:40:29.626176 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.626179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.626189 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:29.626221 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.626222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.626233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.626297 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:29.626509 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.627227 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.628492 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.628949 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.629010 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.629022 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.629146 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.629188 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260806 22:40:29.629251 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.625903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:34757 (local address 127.0.0.1:46092)
0806 22:40:29.626035 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.626046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:29.626101 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:29.626358 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:29.626360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:29.626362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:29.626431 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.626433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.627363 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.627364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.628415 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.628419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.629022 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.629023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.629089 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.629092 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:29.629136 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:40:29.629180 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260806 22:40:29.629519 11576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "298726ce9e304e24bca54ff4f3c82854" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" is_pre_election: true
I20260806 22:40:29.629617 11576 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 298726ce9e304e24bca54ff4f3c82854 in term 1.
I20260806 22:40:29.629765 11261 leader_election.cc:302] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 298726ce9e304e24bca54ff4f3c82854, fbe0b6e8025a413a9f89d58b0b6e491e; no voters: c4ad2cbb63fc4481ad69570692339466
I20260806 22:40:29.629889 12333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:40:29.629940 12333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c4ad2cbb63fc4481ad69570692339466)
I20260806 22:40:29.629959 12333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:40:29.630517 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.627142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:40433 (local address 127.0.0.1:47194)
0806 22:40:29.627332 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.627342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:29.627390 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:29.627585 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:29.627586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:29.627589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:29.627649 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.627650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.628462 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.628463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.629297 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.629299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.630276 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.630277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.630357 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.630361 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:29.630408 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:40:29.630454 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":27,"thread_start_us":49,"threads_started":1}
I20260806 22:40:29.630525 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.627176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.0.186.253:40433)
0806 22:40:29.627485 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.627486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.627493 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:29.627512 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.627513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.627522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.627581 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:29.627711 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.628386 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.629704 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.630197 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.630255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.630264 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.630417 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.630468 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":273,"thread_start_us":78,"threads_started":1}
I20260806 22:40:29.630743 11298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "298726ce9e304e24bca54ff4f3c82854" is_pre_election: true
I20260806 22:40:29.630942 12333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:29.631047 11298 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbe0b6e8025a413a9f89d58b0b6e491e in current term 2: Already voted for candidate 298726ce9e304e24bca54ff4f3c82854 in this term.
I20260806 22:40:29.631062 12333 leader_election.cc:288] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 election: Requested vote from peers c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337), fbe0b6e8025a413a9f89d58b0b6e491e (127.0.186.252:34757)
I20260806 22:40:29.631211 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.631127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45252)
0806 22:40:29.631161 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.631191 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:40:29.631237 11556 leader_election.cc:302] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbe0b6e8025a413a9f89d58b0b6e491e; no voters: 298726ce9e304e24bca54ff4f3c82854, c4ad2cbb63fc4481ad69570692339466
I20260806 22:40:29.631199 11576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "298726ce9e304e24bca54ff4f3c82854" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e"
I20260806 22:40:29.631327 11576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:40:29.631577 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.631499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:45260)
0806 22:40:29.631529 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.631557 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:40:29.631649 11265 leader_election.cc:334] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337): Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
I20260806 22:40:29.632053 11576 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 298726ce9e304e24bca54ff4f3c82854 in term 2.
I20260806 22:40:29.632130 12344 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:29.632198 11261 leader_election.cc:302] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 298726ce9e304e24bca54ff4f3c82854, fbe0b6e8025a413a9f89d58b0b6e491e; no voters: c4ad2cbb63fc4481ad69570692339466
I20260806 22:40:29.632269 12333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:40:29.632393 12333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [term 2 LEADER]: Becoming Leader. State: Replica: 298726ce9e304e24bca54ff4f3c82854, State: Running, Role: LEADER
I20260806 22:40:29.632477 12333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } }
I20260806 22:40:29.632790 12351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 298726ce9e304e24bca54ff4f3c82854. Latest consensus state: current_term: 2 leader_uuid: "298726ce9e304e24bca54ff4f3c82854" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:29.632863 12351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:29.633004 12354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:29.633481 12354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:29.633550 12354 catalog_manager.cc:1259] Loaded cluster ID: 37389e40cc1145b1a630b0c6ba518290
I20260806 22:40:29.633579 12354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:29.633730 12354 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:29.633797 12354 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Loaded TSK: 0
I20260806 22:40:29.633944 12354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:29.688314 11286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:29.689973 11286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:29.691039 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.690920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:56842)
0806 22:40:29.690973 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.691007 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:40:29.691128 11576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [term 2 FOLLOWER]: Refusing update from remote peer 298726ce9e304e24bca54ff4f3c82854: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 22:40:29.691372 12333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 22:40:29.691428 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.691317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.0.1:56848)
0806 22:40:29.691364 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.691403 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:40:29.691514 11265 consensus_peers.cc:487] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 -> Peer c4ad2cbb63fc4481ad69570692339466 (127.0.186.254:46337): Couldn't send request to peer c4ad2cbb63fc4481ad69570692339466. Status: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:40:29.692018 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 298726ce9e304e24bca54ff4f3c82854. Latest consensus state: current_term: 2 leader_uuid: "298726ce9e304e24bca54ff4f3c82854" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:29.692107 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:29.692525 12333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "298726ce9e304e24bca54ff4f3c82854" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:29.692615 12333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:29.692713 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "298726ce9e304e24bca54ff4f3c82854" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c4ad2cbb63fc4481ad69570692339466" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46337 } } peers { permanent_uuid: "298726ce9e304e24bca54ff4f3c82854" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 40433 } attrs { promote: false } } peers { permanent_uuid: "fbe0b6e8025a413a9f89d58b0b6e491e" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 34757 } attrs { promote: false } } }
I20260806 22:40:29.692809 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe0b6e8025a413a9f89d58b0b6e491e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:29.697536 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44893 (local address 127.0.0.1:60952)
0806 22:40:29.694461 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.694470 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:29.694509 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:29.694882 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:29.694883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:29.694885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:29.694928 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.694929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695904 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.695905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.696605 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.696607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697309 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.697374 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.697378 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:29.697430 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:40:29.697467 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":1}
I20260806 22:40:29.697678 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35121 (local address 127.0.0.1:60582)
0806 22:40:29.694406 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.694415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:29.694451 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:29.694709 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:29.694710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:29.694711 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:29.694763 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.694764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695658 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.695659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.696725 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.696727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697481 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.697544 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.697549 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:29.697576 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:40:29.697614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":3}
I20260806 22:40:29.697558 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.0.186.195:44893)
0806 22:40:29.694765 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.694767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.694774 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:29.694799 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.694800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.694810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.694867 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:29.695024 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695796 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.696683 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697231 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697302 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.697314 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.697444 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.697487 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260806 22:40:29.697991 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.0.186.193:35121)
0806 22:40:29.694577 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.694579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.694588 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:29.694615 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.694616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.694626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.694822 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:29.694863 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695584 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.696817 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697393 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697832 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.697848 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.697867 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.697922 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260806 22:40:29.697992 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:40993 (local address 127.0.0.1:53622)
0806 22:40:29.694620 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.694630 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:29.694677 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:29.694937 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:29.694938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:29.694939 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:29.694979 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.694980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695871 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.695872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.696913 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:29.696915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697793 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:29.697858 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.697862 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:29.697887 (+ 25us) client_negotiation.cc:240] Negotiation successful
0806 22:40:29.697926 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":64,"threads_started":1}
I20260806 22:40:29.698143 12140 tablet_service.cc:1467] Processing CreateTablet for tablet 1c45aba5726a48499dbc93fd437d0125 (DEFAULT_TABLE table=test-table [id=879175e83bf24c5eacdc6ac0c5dc6331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:29.698282 12140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c45aba5726a48499dbc93fd437d0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:29.699358 12369 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Bootstrap starting.
I20260806 22:40:29.699388 11752 tablet_service.cc:1467] Processing CreateTablet for tablet 1c45aba5726a48499dbc93fd437d0125 (DEFAULT_TABLE table=test-table [id=879175e83bf24c5eacdc6ac0c5dc6331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:29.699509 11752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c45aba5726a48499dbc93fd437d0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:29.699779 12369 tablet_bootstrap.cc:654] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:29.700275 12369 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: No bootstrap required, opened a new log
I20260806 22:40:29.700321 12369 ts_tablet_manager.cc:1397] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:29.700454 12369 raft_consensus.cc:348] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.700515 12369 raft_consensus.cc:374] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:29.700531 12369 raft_consensus.cc:729] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8633da54eb634804b8ed726e4eee00df, State: Initialized, Role: FOLLOWER
I20260806 22:40:29.700543 12371 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: Bootstrap starting.
I20260806 22:40:29.700594 12369 consensus_queue.cc:260] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.700729 12369 ts_tablet_manager.cc:1428] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:29.700901 12371 tablet_bootstrap.cc:654] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:29.700966 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.694681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.0.186.194:40993)
0806 22:40:29.694799 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.694801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.694809 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:29.694833 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.694833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.694844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.694930 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:29.695056 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.695796 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.697003 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.697697 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.700821 (+ 3124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.700836 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.700856 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.700903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260806 22:40:29.701153 11947 tablet_service.cc:1467] Processing CreateTablet for tablet 1c45aba5726a48499dbc93fd437d0125 (DEFAULT_TABLE table=test-table [id=879175e83bf24c5eacdc6ac0c5dc6331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:29.701257 11947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c45aba5726a48499dbc93fd437d0125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:29.701277 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.700892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:60401)
0806 22:40:29.701212 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.701246 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":75,"threads_started":1}
I20260806 22:40:29.701546 12237 heartbeater.cc:503] Master 127.0.186.253:40433 was elected leader, sending a full tablet report...
I20260806 22:40:29.701738 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.701628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:48453)
0806 22:40:29.701694 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.701717 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:40:29.702440 12378 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Bootstrap starting.
I20260806 22:40:29.702732 12371 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: No bootstrap required, opened a new log
I20260806 22:40:29.702755 12378 tablet_bootstrap.cc:654] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:29.702775 12371 ts_tablet_manager.cc:1397] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:29.702901 12371 raft_consensus.cc:348] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.702960 12371 raft_consensus.cc:374] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:29.702972 12371 raft_consensus.cc:729] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308bc0d3695040e6a296bbb4613fea1d, State: Initialized, Role: FOLLOWER
I20260806 22:40:29.703001 12371 consensus_queue.cc:260] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.703135 12371 ts_tablet_manager.cc:1428] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:29.703521 12378 tablet_bootstrap.cc:492] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: No bootstrap required, opened a new log
I20260806 22:40:29.703576 12378 ts_tablet_manager.cc:1397] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:29.703701 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.703479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:53977)
0806 22:40:29.703620 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.703658 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":100,"threads_started":1}
I20260806 22:40:29.703749 12378 raft_consensus.cc:348] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.703819 12378 raft_consensus.cc:374] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:29.703837 12378 raft_consensus.cc:729] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f860820c864aef8989d22e2e64aa62, State: Initialized, Role: FOLLOWER
I20260806 22:40:29.703872 12378 consensus_queue.cc:260] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.703999 11846 heartbeater.cc:503] Master 127.0.186.253:40433 was elected leader, sending a full tablet report...
I20260806 22:40:29.704016 12378 ts_tablet_manager.cc:1428] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:29.704159 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.704074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:59565)
0806 22:40:29.704107 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.704134 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 22:40:29.704609 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.704390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:37217)
0806 22:40:29.704523 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.704568 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260806 22:40:29.704907 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.704824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:47257)
0806 22:40:29.704854 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.704881 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 22:40:29.704941 12043 heartbeater.cc:503] Master 127.0.186.253:40433 was elected leader, sending a full tablet report...
I20260806 22:40:29.779305 12372 raft_consensus.cc:482] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:29.779407 12372 raft_consensus.cc:504] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.779575 12372 leader_election.cc:288] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121), 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
I20260806 22:40:29.779839 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c45aba5726a48499dbc93fd437d0125" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308bc0d3695040e6a296bbb4613fea1d" is_pre_election: true
I20260806 22:40:29.779845 11972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c45aba5726a48499dbc93fd437d0125" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f860820c864aef8989d22e2e64aa62" is_pre_election: true
I20260806 22:40:29.779958 11972 raft_consensus.cc:2454] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 0.
I20260806 22:40:29.779958 11781 raft_consensus.cc:2454] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 0.
I20260806 22:40:29.780155 12073 leader_election.cc:302] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308bc0d3695040e6a296bbb4613fea1d, 8633da54eb634804b8ed726e4eee00df; no voters:
I20260806 22:40:29.780277 12372 raft_consensus.cc:2781] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:29.780314 12372 raft_consensus.cc:482] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:29.780332 12372 raft_consensus.cc:3037] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:29.781029 12372 raft_consensus.cc:504] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.781155 12372 leader_election.cc:288] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 election: Requested vote from peers 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121), 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993)
I20260806 22:40:29.781315 11972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c45aba5726a48499dbc93fd437d0125" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f860820c864aef8989d22e2e64aa62"
I20260806 22:40:29.781338 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c45aba5726a48499dbc93fd437d0125" candidate_uuid: "8633da54eb634804b8ed726e4eee00df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308bc0d3695040e6a296bbb4613fea1d"
I20260806 22:40:29.781409 11972 raft_consensus.cc:3037] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:29.781409 11781 raft_consensus.cc:3037] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:29.782167 11781 raft_consensus.cc:2454] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 1.
I20260806 22:40:29.782166 11972 raft_consensus.cc:2454] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8633da54eb634804b8ed726e4eee00df in term 1.
I20260806 22:40:29.782351 12073 leader_election.cc:302] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308bc0d3695040e6a296bbb4613fea1d, 8633da54eb634804b8ed726e4eee00df; no voters:
I20260806 22:40:29.782488 12372 raft_consensus.cc:2781] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:29.782670 12372 raft_consensus.cc:686] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 1 LEADER]: Becoming Leader. State: Replica: 8633da54eb634804b8ed726e4eee00df, State: Running, Role: LEADER
I20260806 22:40:29.782725 12372 consensus_queue.cc:237] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } }
I20260806 22:40:29.782989 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.782880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:48013)
0806 22:40:29.782928 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.782955 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:40:29.783476 11286 catalog_manager.cc:5495] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df reported cstate change: term changed from 0 to 1, leader changed from <none> to 8633da54eb634804b8ed726e4eee00df (127.0.186.195). New cstate: current_term: 1 leader_uuid: "8633da54eb634804b8ed726e4eee00df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8633da54eb634804b8ed726e4eee00df" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:29.790191 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.788151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.0.186.195:44893)
0806 22:40:29.788192 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:29.788194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:29.788203 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:29.788268 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:29.788269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:29.788279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:29.788326 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:29.788443 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.788911 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.789606 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:29.789797 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:29.789850 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:29.789984 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:29.790015 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:29.790054 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:29.790103 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:40:29.790125 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:29.790956 11972 raft_consensus.cc:1261] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Refusing update from remote peer 8633da54eb634804b8ed726e4eee00df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:29.790993 11781 raft_consensus.cc:1261] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Refusing update from remote peer 8633da54eb634804b8ed726e4eee00df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:29.791169 12372 consensus_queue.cc:1035] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f860820c864aef8989d22e2e64aa62" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:29.791283 12386 consensus_queue.cc:1035] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [LEADER]: Connected to new peer: Peer: permanent_uuid: "308bc0d3695040e6a296bbb4613fea1d" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:29.791812 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.791715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:33145)
0806 22:40:29.791754 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.791782 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:40:29.792421 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.792194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:33897)
0806 22:40:29.792358 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.792390 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 22:40:29.792755 12393 mvcc.cc:205] Tried to move back new op lower bound from 7315685498022371328 to 7315685497990914048. Current Snapshot: MvccSnapshot[applied={T|T < 7315685498022371328}]
I20260806 22:40:29.792965 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.792845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.195:42489)
0806 22:40:29.792914 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.792940 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:40:29.793529 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.793310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.193:58003)
0806 22:40:29.793474 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.793502 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 22:40:29.793814 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.793724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46337 (local address 127.0.186.194:33437)
0806 22:40:29.793759 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.793787 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 22:40:29.794066 12394 mvcc.cc:205] Tried to move back new op lower bound from 7315685498022371328 to 7315685497990914048. Current Snapshot: MvccSnapshot[applied={T|T < 7315685498022371328}]
I20260806 22:40:29.795336 11283 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:29.795416 11283 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:29.796541 11283 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 298726ce9e304e24bca54ff4f3c82854: Sending DeleteTablet for 3 replicas of tablet 1c45aba5726a48499dbc93fd437d0125
I20260806 22:40:29.796828 12140 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c45aba5726a48499dbc93fd437d0125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:29 UTC) from {username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:60952
I20260806 22:40:29.796856 11752 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c45aba5726a48499dbc93fd437d0125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:29 UTC) from {username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:60582
I20260806 22:40:29.796882 11947 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c45aba5726a48499dbc93fd437d0125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:29 UTC) from {username='oryx', principal='oryx/127.0.186.253@KRBTEST.COM'} at 127.0.0.1:53622
I20260806 22:40:29.797037 12407 tablet_replica.cc:330] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: stopping tablet replica
I20260806 22:40:29.797096 12407 raft_consensus.cc:2229] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:29.797156 12407 raft_consensus.cc:2258] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:29.797222 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 11524
I20260806 22:40:29.797261 12408 tablet_replica.cc:330] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d: stopping tablet replica
I20260806 22:40:29.797307 12408 raft_consensus.cc:2229] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:29.797343 12408 raft_consensus.cc:2258] T 1c45aba5726a48499dbc93fd437d0125 P 308bc0d3695040e6a296bbb4613fea1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:29.809764 12410 tablet_replica.cc:330] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: stopping tablet replica
I20260806 22:40:29.809849 12410 raft_consensus.cc:2229] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:29.809907 12410 raft_consensus.cc:2258] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:29.810837 12407 ts_tablet_manager.cc:1905] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:29.810837 12410 ts_tablet_manager.cc:1905] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:29.812482 12410 ts_tablet_manager.cc:1918] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:29.812546 12410 log.cc:1196] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/1c45aba5726a48499dbc93fd437d0125
I20260806 22:40:29.812549 12407 ts_tablet_manager.cc:1918] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:29.812609 12407 log.cc:1196] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/1c45aba5726a48499dbc93fd437d0125
I20260806 22:40:29.812764 12410 ts_tablet_manager.cc:1939] T 1c45aba5726a48499dbc93fd437d0125 P 91f860820c864aef8989d22e2e64aa62: Deleting consensus metadata
I20260806 22:40:29.812886 12407 ts_tablet_manager.cc:1939] T 1c45aba5726a48499dbc93fd437d0125 P 8633da54eb634804b8ed726e4eee00df: Deleting consensus metadata
I20260806 22:40:29.813071 11258 catalog_manager.cc:4841] TS 91f860820c864aef8989d22e2e64aa62 (127.0.186.194:40993): tablet 1c45aba5726a48499dbc93fd437d0125 (table test-table [id=879175e83bf24c5eacdc6ac0c5dc6331]) successfully deleted
I20260806 22:40:29.813185 11258 catalog_manager.cc:4841] TS 8633da54eb634804b8ed726e4eee00df (127.0.186.195:44893): tablet 1c45aba5726a48499dbc93fd437d0125 (table test-table [id=879175e83bf24c5eacdc6ac0c5dc6331]) successfully deleted
I20260806 22:40:29.815111 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 11853
I20260806 22:40:29.815126 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:29.815012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:35121 (local address 127.0.0.1:60596)
0806 22:40:29.815064 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:29.815093 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:40:29.817091 11261 catalog_manager.cc:4584] TS 308bc0d3695040e6a296bbb4613fea1d (127.0.186.193:35121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c45aba5726a48499dbc93fd437d0125: Network error: Client connection negotiation failed: client connection to 127.0.186.193:35121: connect: Connection refused (error 111)
I20260806 22:40:29.828673 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 12049
I20260806 22:40:29.836171 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 11092
I20260806 22:40:29.844841 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 11367
2026-08-06T22:40:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5207 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:29 dist-test-slave-gjrz krb5kdc[12414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:29 dist-test-slave-gjrz krb5kdc[12414](info): set up 2 sockets
Aug 06 22:40:29 dist-test-slave-gjrz krb5kdc[12414](info): commencing operation
krb5kdc: starting...
W20260806 22:40:31.009959 747 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab.
Entry for principal oryx/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab.
I20260806 22:40:31.128103 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40381
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab
--principal=oryx/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:40381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:31.208864 12428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:31.209062 12428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:31.209084 12428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:31.210175 12428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:31.210206 12428 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:40:31.210218 12428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:31.210229 12428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:31.210248 12428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:31.210264 12428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:31.211709 12428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:40381
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.254.keytab
--principal=oryx/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:31.211887 12428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:31.212064 12428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:31.216504 12442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.216471 12443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.216491 12446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:31.216693 12428 server_base.cc:1034] running on GCE node
I20260806 22:40:31.216895 12428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:31.217149 12428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:31.218278 12428 hybrid_clock.cc:648] HybridClock initialized: now 1786056031218265 us; error 30 us; skew 500 ppm
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:31.220644 12428 init.cc:377] Logged in from keytab as oryx/127.0.186.254@KRBTEST.COM (short username oryx)
I20260806 22:40:31.221030 12428 webserver.cc:460] Webserver started at http://127.0.186.254:43911/ using document root <none> and password file <none>
I20260806 22:40:31.221237 12428 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:31.221282 12428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:31.221407 12428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:31.222396 12428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "3f25d80326bd4810b8256da83d53a608"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "49fd51086243944d22a16f5a1b2fabb5"
server_key_iv: "7fed37a7bce80bd4b1e74090816dedb1"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.222738 12428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "3f25d80326bd4810b8256da83d53a608"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "49fd51086243944d22a16f5a1b2fabb5"
server_key_iv: "7fed37a7bce80bd4b1e74090816dedb1"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.223978 12428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:31.224596 12456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.224748 12428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:31.224828 12428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "3f25d80326bd4810b8256da83d53a608"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "49fd51086243944d22a16f5a1b2fabb5"
server_key_iv: "7fed37a7bce80bd4b1e74090816dedb1"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.224879 12428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:31.251313 12428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:31.253377 12428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:31.253485 12428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:31.257817 12428 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:40381
I20260806 22:40:31.257899 12551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:40381 every 8 connection(s)
I20260806 22:40:31.258183 12428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:31.258796 12552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.261113 12552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Bootstrap starting.
I20260806 22:40:31.261656 12552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.261945 12552 log.cc:824] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:31.262559 12552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: No bootstrap required, opened a new log
I20260806 22:40:31.262928 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 12428
I20260806 22:40:31.263031 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:31.263203 747 external_mini_cluster.cc:1428] Setting key 63d77b224869be67088b45703105819f
I20260806 22:40:31.265026 12552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } }
I20260806 22:40:31.265205 12552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.265242 12552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f25d80326bd4810b8256da83d53a608, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.265383 12552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } }
I20260806 22:40:31.265456 12552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:31.265478 12552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:31.265524 12552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:31.266117 12552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } }
I20260806 22:40:31.266224 12552 leader_election.cc:302] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f25d80326bd4810b8256da83d53a608; no voters:
I20260806 22:40:31.266378 12552 leader_election.cc:288] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:31.266423 12558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:31.266567 12558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [term 1 LEADER]: Becoming Leader. State: Replica: 3f25d80326bd4810b8256da83d53a608, State: Running, Role: LEADER
I20260806 22:40:31.266613 12552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:31.266657 12558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } }
I20260806 22:40:31.267001 12559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f25d80326bd4810b8256da83d53a608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } } }
I20260806 22:40:31.267097 12559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:31.267000 12560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f25d80326bd4810b8256da83d53a608. Latest consensus state: current_term: 1 leader_uuid: "3f25d80326bd4810b8256da83d53a608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f25d80326bd4810b8256da83d53a608" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40381 } } }
I20260806 22:40:31.267283 12560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:31.267315 12564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:31.267807 12564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:31.269847 12564 catalog_manager.cc:1347] Generated new cluster ID: a9ca821b82d34304b0be03d918249de6
I20260806 22:40:31.269901 12564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:31.274798 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.263680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.0.186.254:40381)
0806 22:40:31.263862 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.263871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.263906 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:40:31.264997 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.265006 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.265069 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.265218 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.265859 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.267243 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.268279 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.268716 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.268811 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.271259 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.271287 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.271288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.271299 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.273431 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.273744 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.273747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.273747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.273843 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.274039 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.274040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.274041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.274260 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.274414 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.274485 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.274628 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260806 22:40:31.284011 12564 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:31.284555 12564 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:31.291488 12564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Generated new TSK 0
I20260806 22:40:31.291661 12564 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
I20260806 22:40:31.324522 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40381
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:31.417059 12584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:31.417249 12584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:31.417271 12584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:31.418370 12584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:31.418408 12584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:31.418428 12584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:31.418458 12584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:31.419951 12584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:31.420169 12584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:31.420372 12584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:31.425038 12593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.425081 12595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.425048 12592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:31.425277 12584 server_base.cc:1034] running on GCE node
I20260806 22:40:31.425483 12584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:31.425709 12584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:31.426862 12584 hybrid_clock.cc:648] HybridClock initialized: now 1786056031426851 us; error 29 us; skew 500 ppm
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:31.429366 12584 init.cc:377] Logged in from keytab as oryx/127.0.186.193@KRBTEST.COM (short username oryx)
I20260806 22:40:31.429762 12584 webserver.cc:460] Webserver started at http://127.0.186.193:42259/ using document root <none> and password file <none>
I20260806 22:40:31.429962 12584 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:31.430009 12584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:31.430135 12584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:31.431137 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "eb208dd802d2b513bcbb30b22dea5b5a"
server_key_iv: "0c5d64934878cf61fc2c0dee9a1f85bc"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.431504 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "eb208dd802d2b513bcbb30b22dea5b5a"
server_key_iv: "0c5d64934878cf61fc2c0dee9a1f85bc"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.432752 12584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:31.433468 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.433640 12584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.433722 12584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "eb208dd802d2b513bcbb30b22dea5b5a"
server_key_iv: "0c5d64934878cf61fc2c0dee9a1f85bc"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.433790 12584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:31.449890 12584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:31.452001 12584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:31.452143 12584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:31.452626 12584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:31.452667 12584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.452695 12584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:31.452713 12584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:31.459241 12584 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:40883
I20260806 22:40:31.459373 12775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:40883 every 8 connection(s)
I20260806 22:40:31.459610 12584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:31.460309 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 12584
I20260806 22:40:31.460418 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:31.460630 747 external_mini_cluster.cc:1428] Setting key c10aa7f228f89f3996911a9807c07170
I20260806 22:40:31.462324 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.453163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:33501 (local address 127.0.186.254:40381)
0806 22:40:31.453225 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.453227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.453372 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:40:31.453886 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.453888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.453900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.453959 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.454813 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.455613 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.456629 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.456929 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.456993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.459510 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.459511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.459511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.459518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.461021 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.461469 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.461470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.461470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.461547 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.461737 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.461738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.461738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.461860 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.461942 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.462155 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.462243 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:40:31.462387 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.453058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.193:33501)
0806 22:40:31.453230 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.453283 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.453778 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.454073 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.454078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.454425 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.454727 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.454734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.455710 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.455711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.456532 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.456536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.457065 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.457066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.457165 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.457622 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.457632 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.459410 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.461105 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.461112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.461115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.461401 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.461600 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.461600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.461601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.461689 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.461931 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.461933 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.462037 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.462117 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.462235 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260806 22:40:31.462921 12776 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40381
I20260806 22:40:31.463052 12776 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:31.463295 12776 heartbeater.cc:511] Master 127.0.186.254:40381 requested a full tablet report, sending...
I20260806 22:40:31.463886 12477 ts_manager.cc:194] Registered new tserver with Master: a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883)
I20260806 22:40:31.464546 12477 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:33501
I20260806 22:40:31.469807 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.463939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.193:53593)
0806 22:40:31.464069 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.464079 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.464130 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.464867 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.464868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.465031 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.465087 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.465089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.465871 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.465871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.466334 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.466335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.466752 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.466753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.466816 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.467219 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.467223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.467539 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.469016 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.469017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.469017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.469170 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.469346 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.469346 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.469347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.469389 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.469603 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.469603 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.469636 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.469677 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.469746 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260806 22:40:31.469858 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.464707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:53593 (local address 127.0.186.254:40381)
0806 22:40:31.464761 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.464763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.464770 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:31.464789 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.464789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.464799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.464850 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.465161 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.465791 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.466402 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.466683 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.466753 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.467614 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.467615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.467616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.467623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.468937 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.469231 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.469232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.469232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.469290 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.469424 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.469425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.469425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.469550 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.469614 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.469703 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.469771 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for oryx/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
I20260806 22:40:31.487249 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40381
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:31.569252 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:31.569447 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:31.569468 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:31.570555 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:31.570588 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:31.570606 12789 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:31.570627 12789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:31.572119 12789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:31.572311 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:31.572499 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:31.577180 12796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.577152 12798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.577153 12795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:31.577322 12789 server_base.cc:1034] running on GCE node
I20260806 22:40:31.577574 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:31.577793 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:31.578928 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786056031578920 us; error 27 us; skew 500 ppm
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:31.581363 12789 init.cc:377] Logged in from keytab as oryx/127.0.186.194@KRBTEST.COM (short username oryx)
I20260806 22:40:31.581784 12789 webserver.cc:460] Webserver started at http://127.0.186.194:40841/ using document root <none> and password file <none>
I20260806 22:40:31.581990 12789 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:31.582037 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:31.582125 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:31.583091 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "c9135210befd499499300995f1386a49"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "5475092d8d58ecf8b9c6c730688ccad4"
server_key_iv: "6fb2664006c79d61b68fe16ae0fb4807"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.583413 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "c9135210befd499499300995f1386a49"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "5475092d8d58ecf8b9c6c730688ccad4"
server_key_iv: "6fb2664006c79d61b68fe16ae0fb4807"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.584553 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:31.585172 12812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.585335 12789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:31.585415 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "c9135210befd499499300995f1386a49"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "5475092d8d58ecf8b9c6c730688ccad4"
server_key_iv: "6fb2664006c79d61b68fe16ae0fb4807"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.585469 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:31.592680 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:31.594424 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:31.594528 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:31.594950 12789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:31.594988 12789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.595006 12789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:31.595017 12789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.601109 12789 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:45713
I20260806 22:40:31.601162 12977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:45713 every 8 connection(s)
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:31.601490 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:31.601907 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 12789
I20260806 22:40:31.602039 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:31.602276 747 external_mini_cluster.cc:1428] Setting key 7e5f2307a772c6d293eced1a42a6e0fe
I20260806 22:40:31.604341 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.595402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:39027 (local address 127.0.186.254:40381)
0806 22:40:31.595478 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.595480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.595665 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:40:31.596156 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.596157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.596172 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.596250 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.597045 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.597826 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.598949 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.599243 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.599336 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.601703 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.601705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.601705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.601714 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.603084 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.603518 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.603518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.603519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.603568 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.603807 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.603808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.603808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.603899 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.603941 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.604220 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.604279 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:31.604552 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.595378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.194:39027)
0806 22:40:31.595542 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.595580 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.596039 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.596317 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.596322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.596675 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.596946 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.596952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.597934 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.597935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.598860 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.598863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.599418 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.599419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.599574 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.600022 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.600031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.601611 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.603154 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.603161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.603163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.603449 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.603623 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.603624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.603624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.603755 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.603945 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.603950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.604103 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.604207 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.604356 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260806 22:40:31.605067 12978 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40381
I20260806 22:40:31.605149 12978 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:31.605314 12978 heartbeater.cc:511] Master 127.0.186.254:40381 requested a full tablet report, sending...
I20260806 22:40:31.605667 12478 ts_manager.cc:194] Registered new tserver with Master: c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:31.606102 12478 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:39027
I20260806 22:40:31.611415 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.606092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:42903 (local address 127.0.186.254:40381)
0806 22:40:31.606125 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.606126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.606236 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:40:31.606292 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.606293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.606302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.606353 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.606678 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.607198 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.608167 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.608340 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.608391 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.609245 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.609246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.609247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.609253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.610391 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.610738 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.610739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.610739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.610781 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.610932 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.610932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.610933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.611077 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.611127 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.611300 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.611360 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:40:31.611476 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.606039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.194:42903)
0806 22:40:31.606181 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.606191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.606247 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.606357 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.606358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.606529 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.606597 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.606599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.607319 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.607320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.608090 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.608093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.608401 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.608402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.608467 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.608814 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.608819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.609159 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.610471 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.610472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.610472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.610677 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.610822 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.610822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.610823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.610881 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.611142 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.611142 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.611186 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.611279 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.611378 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
I20260806 22:40:31.628186 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:40381
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:31.708874 12991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:31.709056 12991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:31.709085 12991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:31.710163 12991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:31.710198 12991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:31.710216 12991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:40:31.710232 12991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:31.711747 12991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.186.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:31.711926 12991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:31.712108 12991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:31.716629 13004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.716638 12999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:31.716681 12998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:31.716830 12991 server_base.cc:1034] running on GCE node
I20260806 22:40:31.717053 12991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:31.717262 12991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:31.718400 12991 hybrid_clock.cc:648] HybridClock initialized: now 1786056031718384 us; error 28 us; skew 500 ppm
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:31.720758 12991 init.cc:377] Logged in from keytab as oryx/127.0.186.195@KRBTEST.COM (short username oryx)
I20260806 22:40:31.721174 12991 webserver.cc:460] Webserver started at http://127.0.186.195:33577/ using document root <none> and password file <none>
I20260806 22:40:31.721382 12991 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:31.721428 12991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:31.721550 12991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:31.722518 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "1a4526fa24544f3aae67dffb540df3f7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "e7f03dce49eadf95d36b8a3ced71facb"
server_key_iv: "9b0f79f707adf0c51959ade3a43d9f8c"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.722832 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "1a4526fa24544f3aae67dffb540df3f7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "e7f03dce49eadf95d36b8a3ced71facb"
server_key_iv: "9b0f79f707adf0c51959ade3a43d9f8c"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.724150 12991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:31.724857 13015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.725009 12991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:31.725108 12991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "1a4526fa24544f3aae67dffb540df3f7"
format_stamp: "Formatted at 2026-08-06 22:40:31 on dist-test-slave-gjrz"
server_key: "e7f03dce49eadf95d36b8a3ced71facb"
server_key_iv: "9b0f79f707adf0c51959ade3a43d9f8c"
server_key_version: "encryptionkey@0"
I20260806 22:40:31.725169 12991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:31.738065 12991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:31.739836 12991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:31.739964 12991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:31.740437 12991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:31.740477 12991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.740504 12991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:31.740521 12991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:31.747100 12991 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:46271
I20260806 22:40:31.747172 13195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:46271 every 8 connection(s)
I20260806 22:40:31.747483 12991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:31.749450 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.740794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:53705 (local address 127.0.186.254:40381)
0806 22:40:31.740850 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.740852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.741062 (+ 210us) server_negotiation.cc:407] Connection header received
0806 22:40:31.741548 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.741549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.741562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.741637 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.742270 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.743021 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.744107 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.744320 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.744378 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.747017 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.747018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.747018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.747027 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.748451 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.748782 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.748783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.748783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.748831 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.748992 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.748992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.748992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.749089 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.749133 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.749328 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.749387 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:31.749604 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.740799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.195:53705)
0806 22:40:31.740945 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.740979 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.741448 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.741714 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.741718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.742005 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.742191 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.742196 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.743110 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.743111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.744015 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.744018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.744397 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.744399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.744529 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.744995 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.745004 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.746926 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.748520 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.748523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.748525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.748724 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.748869 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.748869 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.748870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.748956 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.749130 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.749133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.749237 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.749313 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.749437 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260806 22:40:31.750258 13196 heartbeater.cc:347] Connected to a master server at 127.0.186.254:40381
I20260806 22:40:31.750346 13196 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:31.750521 13196 heartbeater.cc:511] Master 127.0.186.254:40381 requested a full tablet report, sending...
I20260806 22:40:31.750936 12478 ts_manager.cc:194] Registered new tserver with Master: 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271)
I20260806 22:40:31.751407 12478 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:53705
I20260806 22:40:31.752971 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 12991
I20260806 22:40:31.753078 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:31.753296 747 external_mini_cluster.cc:1428] Setting key cdda17e463c0f5bff941a016c75bd0e1
I20260806 22:40:31.754068 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:31.756808 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.751316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:44887 (local address 127.0.186.254:40381)
0806 22:40:31.751354 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.751355 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.751411 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:40:31.751512 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.751513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.751524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.751575 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.751972 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.752468 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.753161 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.753432 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.753506 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.754344 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.754345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.754345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.754353 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.755729 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.756037 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.756038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.756038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.756098 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.756326 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.756326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.756327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.756460 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.756538 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.756689 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.756750 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:31.756876 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.751216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.186.195:44887)
0806 22:40:31.751355 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.751364 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.751426 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.751696 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.751697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.751845 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:31.751898 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.751899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.752550 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.752550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.753089 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.753091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.753507 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.753508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.753576 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.753945 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:31.753950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:31.754265 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:31.755807 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.755808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.755808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.755972 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.756191 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:31.756192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:31.756193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:31.756258 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:31.756540 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:31.756541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:31.756593 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.756673 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.756778 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:40:31.775945 12556 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:31.770960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.0.186.254:40381)
0806 22:40:31.771038 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.771039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.771047 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:31.771069 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.771070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.771078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.771125 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.771685 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.772169 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.772853 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.773069 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.773134 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.775871 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44114: BlockingRecv error: recv got EOF from 127.0.0.1:44114 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 22:40:31 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:31.782603 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.776612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.0.186.254:40381)
0806 22:40:31.776660 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.776662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.776706 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:40:31.776753 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.776754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.776761 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.776804 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:31.777387 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.777844 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.778460 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.778637 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.778683 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.780643 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:31.780645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:31.780645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:31.780651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:31.781744 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.782036 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.782036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.782037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.782092 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:31.782231 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:31.782232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:31.782232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:31.782321 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:31.782385 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.782454 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.782527 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:31.784114 12478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:31.785593 12478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:31.792541 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.787776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.0.1:59458)
0806 22:40:31.787937 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.788014 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.788074 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.788505 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.788509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.788511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:31.788605 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.788608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.789904 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.789905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.791308 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.791309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.792217 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.792218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.792307 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.792324 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.792400 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.792453 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":38,"thread_start_us":55,"threads_started":1}
I20260806 22:40:31.792743 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.788134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59458 (local address 127.0.186.193:40883)
0806 22:40:31.788324 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.788329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.788340 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:31.788381 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.788383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.788397 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.788685 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:31.788749 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.789811 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.791375 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.792142 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.792215 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.792305 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.792427 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.792468 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260806 22:40:31.793296 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.787962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:45713 (local address 127.0.0.1:49232)
0806 22:40:31.788586 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.788596 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.788638 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.790104 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.790105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.790107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:31.790195 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.790197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.791109 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.791110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.792201 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.792205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.793069 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.793070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.793132 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.793136 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.793188 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.793228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":45,"thread_start_us":36,"threads_started":1}
I20260806 22:40:31.793344 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.788130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49232 (local address 127.0.186.194:45713)
0806 22:40:31.789963 (+ 1833us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.789967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.789982 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:40:31.790012 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.790014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.790026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.790090 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:31.790257 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.791034 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.792279 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.792982 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.793050 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.793123 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.793206 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.793243 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1794,"thread_start_us":96,"threads_started":1}
I20260806 22:40:31.793466 12691 tablet_service.cc:1467] Processing CreateTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (DEFAULT_TABLE table=test-table [id=34cde42d010e4afd93722d7399d6b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.793896 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.787772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46271 (local address 127.0.0.1:53862)
0806 22:40:31.787965 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.787983 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.788078 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.788505 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.788509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.788513 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:31.788605 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.788608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.789892 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.789893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.791301 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.791305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.793692 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.793693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.793758 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.793762 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.793797 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.793833 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":67,"thread_start_us":56,"threads_started":1}
I20260806 22:40:31.793885 12691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.794020 12882 tablet_service.cc:1467] Processing CreateTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (DEFAULT_TABLE table=test-table [id=34cde42d010e4afd93722d7399d6b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.794234 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.787923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.0.186.195:46271)
0806 22:40:31.788349 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.788354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.788365 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:31.788402 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.788404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.788419 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.788497 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:31.788693 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.789817 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.792340 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.793590 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.793658 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.793783 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.793820 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.793857 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":82,"threads_started":1}
I20260806 22:40:31.794399 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.794930 13092 tablet_service.cc:1467] Processing CreateTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (DEFAULT_TABLE table=test-table [id=34cde42d010e4afd93722d7399d6b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.795326 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.797483 13238 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Bootstrap starting.
I20260806 22:40:31.797853 13239 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Bootstrap starting.
I20260806 22:40:31.798213 13238 tablet_bootstrap.cc:654] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.798493 13239 tablet_bootstrap.cc:654] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.798498 13238 log.cc:824] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:31.798744 13241 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Bootstrap starting.
I20260806 22:40:31.798823 13239 log.cc:824] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:31.799383 13238 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: No bootstrap required, opened a new log
I20260806 22:40:31.799425 13241 tablet_bootstrap.cc:654] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.799460 13238 ts_tablet_manager.cc:1397] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:31.799562 13239 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: No bootstrap required, opened a new log
I20260806 22:40:31.799628 13239 ts_tablet_manager.cc:1397] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:31.799749 13241 log.cc:824] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:31.800496 13241 tablet_bootstrap.cc:492] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: No bootstrap required, opened a new log
I20260806 22:40:31.800563 13241 ts_tablet_manager.cc:1397] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:31.801128 13239 raft_consensus.cc:348] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.801257 13239 raft_consensus.cc:374] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.801288 13239 raft_consensus.cc:729] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.801309 13238 raft_consensus.cc:348] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.801398 13239 consensus_queue.cc:260] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.801468 13238 raft_consensus.cc:374] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.801509 13238 raft_consensus.cc:729] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9135210befd499499300995f1386a49, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.801608 13238 consensus_queue.cc:260] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.801657 13239 ts_tablet_manager.cc:1428] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:31.801880 13238 ts_tablet_manager.cc:1428] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:31.801914 12978 heartbeater.cc:503] Master 127.0.186.254:40381 was elected leader, sending a full tablet report...
I20260806 22:40:31.802031 12776 heartbeater.cc:503] Master 127.0.186.254:40381 was elected leader, sending a full tablet report...
I20260806 22:40:31.802435 13241 raft_consensus.cc:348] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.802591 13241 raft_consensus.cc:374] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.802615 13241 raft_consensus.cc:729] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.802702 13241 consensus_queue.cc:260] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.802956 13241 ts_tablet_manager.cc:1428] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:40:31.803119 13196 heartbeater.cc:503] Master 127.0.186.254:40381 was elected leader, sending a full tablet report...
W20260806 22:40:31.852123 12979 tablet.cc:2369] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:31.932232 13249 raft_consensus.cc:482] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:31.932359 13249 raft_consensus.cc:504] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.932756 13249 leader_election.cc:288] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271), a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883)
I20260806 22:40:31.935719 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.933034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:40739 (local address 127.0.186.195:46271)
0806 22:40:31.933083 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.933085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.933098 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:40:31.933157 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.933158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.933167 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.933217 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:31.933380 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.934056 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.934979 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.935382 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.935445 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.935455 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.935616 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.935649 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:31.935730 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.932964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46271 (local address 127.0.186.194:40739)
0806 22:40:31.933026 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.933038 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.933085 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.933239 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.933240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.933241 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:31.933317 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.933319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.934130 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.934131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.934872 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.934874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.935457 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.935459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.935556 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.935559 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.935605 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.935646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:40:31.936003 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.933035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:33553 (local address 127.0.186.193:40883)
0806 22:40:31.933092 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.933094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.933211 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:40:31.933259 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.933260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.933268 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.933318 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:31.933516 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.934118 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.934984 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.935398 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.935467 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.935475 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.935883 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.935927 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:40:31.936039 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f06c7eb9cb44f388f5a67b7cecf2a58" candidate_uuid: "c9135210befd499499300995f1386a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7" is_pre_election: true
I20260806 22:40:31.936211 13129 raft_consensus.cc:2454] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9135210befd499499300995f1386a49 in term 0.
I20260806 22:40:31.936259 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.932965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.186.194:33553)
0806 22:40:31.933150 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:31.933158 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:31.933206 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:31.933325 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:31.933326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:31.933339 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:31.933427 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.933428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.934191 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.934193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.934895 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:31.934897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.935467 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.935468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:31.935781 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.935786 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:31.936094 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 22:40:31.936180 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":13,"thread_start_us":112,"threads_started":1}
I20260806 22:40:31.936441 12813 leader_election.cc:302] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:31.936581 13249 raft_consensus.cc:2781] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:31.936628 13249 raft_consensus.cc:482] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:31.936646 13249 raft_consensus.cc:3037] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:31.936610 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f06c7eb9cb44f388f5a67b7cecf2a58" candidate_uuid: "c9135210befd499499300995f1386a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" is_pre_election: true
I20260806 22:40:31.936756 12718 raft_consensus.cc:2454] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9135210befd499499300995f1386a49 in term 0.
I20260806 22:40:31.937436 13249 raft_consensus.cc:504] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.937574 13249 leader_election.cc:288] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [CANDIDATE]: Term 1 election: Requested vote from peers 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271), a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883)
I20260806 22:40:31.937687 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f06c7eb9cb44f388f5a67b7cecf2a58" candidate_uuid: "c9135210befd499499300995f1386a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7"
I20260806 22:40:31.937727 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f06c7eb9cb44f388f5a67b7cecf2a58" candidate_uuid: "c9135210befd499499300995f1386a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
I20260806 22:40:31.937760 13129 raft_consensus.cc:3037] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:31.937800 12718 raft_consensus.cc:3037] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:31.938431 13129 raft_consensus.cc:2454] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9135210befd499499300995f1386a49 in term 1.
I20260806 22:40:31.938441 12718 raft_consensus.cc:2454] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9135210befd499499300995f1386a49 in term 1.
I20260806 22:40:31.938612 12815 leader_election.cc:302] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fa579b2b59437ea6b76e3baaa289b7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:31.938686 13249 raft_consensus.cc:2781] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:31.938813 13249 raft_consensus.cc:686] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 1 LEADER]: Becoming Leader. State: Replica: c9135210befd499499300995f1386a49, State: Running, Role: LEADER
I20260806 22:40:31.938897 13249 consensus_queue.cc:237] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:31.939711 12478 catalog_manager.cc:5495] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9135210befd499499300995f1386a49 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "c9135210befd499499300995f1386a49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } health_report { overall_health: HEALTHY } } }
W20260806 22:40:31.960574 12777 tablet.cc:2369] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:31.973959 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:31.971817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.0.186.194:45713)
0806 22:40:31.971864 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:31.971866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:31.971873 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:31.971922 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:31.971922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:31.971940 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:31.972018 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:31.972110 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.972559 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.973261 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:31.973451 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:31.973503 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:31.973628 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:31.973734 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:31.973793 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:31.973886 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:40:31.973904 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:31.975752 12718 raft_consensus.cc:1261] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Refusing update from remote peer c9135210befd499499300995f1386a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:31.975754 13129 raft_consensus.cc:1261] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Refusing update from remote peer c9135210befd499499300995f1386a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:31.976064 13259 consensus_queue.cc:1035] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:31.976181 13249 consensus_queue.cc:1035] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:31.978343 13265 mvcc.cc:205] Tried to move back new op lower bound from 7315685506970058752 to 7315685506822266880. Current Snapshot: MvccSnapshot[applied={T|T < 7315685506970058752}]
I20260806 22:40:31.978483 13268 mvcc.cc:205] Tried to move back new op lower bound from 7315685506970058752 to 7315685506822266880. Current Snapshot: MvccSnapshot[applied={T|T < 7315685506970058752}]
I20260806 22:40:31.979072 13274 mvcc.cc:205] Tried to move back new op lower bound from 7315685506970058752 to 7315685506822266880. Current Snapshot: MvccSnapshot[applied={T|T < 7315685506970058752 or (T in {7315685506970058752})}]
I20260806 22:40:31.980484 12478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:31.980577 12478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:31.981288 12478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Sending DeleteTablet for 3 replicas of tablet 8f06c7eb9cb44f388f5a67b7cecf2a58
I20260806 22:40:31.981612 12691 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:31 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260806 22:40:31.981611 13092 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:31 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:53862
I20260806 22:40:31.981611 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:31 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49232
I20260806 22:40:31.981895 13285 tablet_replica.cc:330] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: stopping tablet replica
I20260806 22:40:31.982036 13285 raft_consensus.cc:2229] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:31.982059 12478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:40:31.982147 13285 raft_consensus.cc:2258] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:40:31.982198 12478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:31.982287 13288 tablet_replica.cc:330] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: stopping tablet replica
I20260806 22:40:31.982376 13288 raft_consensus.cc:2229] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:31.982717 13288 raft_consensus.cc:2258] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:31.983235 13285 ts_tablet_manager.cc:1905] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:31.983496 12882 tablet_service.cc:1467] Processing CreateTablet for tablet 52fdf08ff3fc430a95e012a259717066 (DEFAULT_TABLE table=test-table [id=942179defef349a49af1513749882c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.983558 12691 tablet_service.cc:1467] Processing CreateTablet for tablet 52fdf08ff3fc430a95e012a259717066 (DEFAULT_TABLE table=test-table [id=942179defef349a49af1513749882c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.983614 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fdf08ff3fc430a95e012a259717066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.983654 12691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fdf08ff3fc430a95e012a259717066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.984730 13239 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Bootstrap starting.
I20260806 22:40:31.984923 13238 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Bootstrap starting.
I20260806 22:40:31.985031 13288 ts_tablet_manager.cc:1905] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:31.985070 13239 tablet_bootstrap.cc:654] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.985356 13238 tablet_bootstrap.cc:654] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.985757 13092 tablet_service.cc:1467] Processing CreateTablet for tablet 52fdf08ff3fc430a95e012a259717066 (DEFAULT_TABLE table=test-table [id=942179defef349a49af1513749882c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:31.985885 13285 ts_tablet_manager.cc:1918] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:31.985903 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fdf08ff3fc430a95e012a259717066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:31.986006 13285 log.cc:1196] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/8f06c7eb9cb44f388f5a67b7cecf2a58
I20260806 22:40:31.986318 13285 ts_tablet_manager.cc:1939] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting consensus metadata
I20260806 22:40:31.986557 13288 ts_tablet_manager.cc:1918] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:31.986630 13288 log.cc:1196] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/8f06c7eb9cb44f388f5a67b7cecf2a58
I20260806 22:40:31.986851 13288 ts_tablet_manager.cc:1939] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P 1a4526fa24544f3aae67dffb540df3f7: Deleting consensus metadata
I20260806 22:40:31.987136 13241 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Bootstrap starting.
I20260806 22:40:31.987217 12457 catalog_manager.cc:4841] TS 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271): tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (table test-table [id=34cde42d010e4afd93722d7399d6b4fb]) successfully deleted
I20260806 22:40:31.987278 12459 catalog_manager.cc:4841] TS a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883): tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (table test-table [id=34cde42d010e4afd93722d7399d6b4fb]) successfully deleted
I20260806 22:40:31.987546 13241 tablet_bootstrap.cc:654] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:31.987740 13239 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: No bootstrap required, opened a new log
I20260806 22:40:31.987782 13239 ts_tablet_manager.cc:1397] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:40:31.987893 13238 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: No bootstrap required, opened a new log
I20260806 22:40:31.987941 13238 ts_tablet_manager.cc:1397] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:40:31.988094 13238 raft_consensus.cc:348] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.988095 13239 raft_consensus.cc:348] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.988164 13238 raft_consensus.cc:374] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.988164 13239 raft_consensus.cc:374] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.988185 13239 raft_consensus.cc:729] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.988185 13238 raft_consensus.cc:729] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9135210befd499499300995f1386a49, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.988224 13239 consensus_queue.cc:260] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.988224 13238 consensus_queue.cc:260] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.988310 13239 ts_tablet_manager.cc:1428] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:31.988310 13238 ts_tablet_manager.cc:1428] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.988865 13241 tablet_bootstrap.cc:492] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: No bootstrap required, opened a new log
I20260806 22:40:31.988914 13241 ts_tablet_manager.cc:1397] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:31.989058 13241 raft_consensus.cc:348] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.989120 13241 raft_consensus.cc:374] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:31.989138 13241 raft_consensus.cc:729] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Initialized, Role: FOLLOWER
I20260806 22:40:31.989182 13241 consensus_queue.cc:260] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:31.989259 13241 ts_tablet_manager.cc:1428] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:31.991376 13286 tablet_replica.cc:330] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: stopping tablet replica
I20260806 22:40:31.991464 13286 raft_consensus.cc:2229] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:31.991587 13286 raft_consensus.cc:2258] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:31.991918 13286 ts_tablet_manager.cc:1905] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:31.993304 13286 ts_tablet_manager.cc:1918] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:31.993396 13286 log.cc:1196] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/8f06c7eb9cb44f388f5a67b7cecf2a58
I20260806 22:40:31.993645 13286 ts_tablet_manager.cc:1939] T 8f06c7eb9cb44f388f5a67b7cecf2a58 P c9135210befd499499300995f1386a49: Deleting consensus metadata
I20260806 22:40:31.993981 12459 catalog_manager.cc:4841] TS c9135210befd499499300995f1386a49 (127.0.186.194:45713): tablet 8f06c7eb9cb44f388f5a67b7cecf2a58 (table test-table [id=34cde42d010e4afd93722d7399d6b4fb]) successfully deleted
W20260806 22:40:31.998003 13197 tablet.cc:2369] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:32.230111 13248 raft_consensus.cc:482] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:32.230199 13248 raft_consensus.cc:504] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.230530 13248 leader_election.cc:288] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9135210befd499499300995f1386a49 (127.0.186.194:45713), 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271)
I20260806 22:40:32.233421 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.230770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:46483 (local address 127.0.186.195:46271)
0806 22:40:32.230820 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.230822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.230975 (+ 153us) server_negotiation.cc:407] Connection header received
0806 22:40:32.231029 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.231030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.231040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.231114 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:32.231252 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.231776 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.232756 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.233115 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.233176 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.233184 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.233323 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.233355 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:32.233440 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.230806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:37299 (local address 127.0.186.194:45713)
0806 22:40:32.230855 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.230857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.230951 (+ 94us) server_negotiation.cc:407] Connection header received
0806 22:40:32.231010 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.231011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.231025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.231072 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:32.231213 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.231731 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.232762 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.233126 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.233187 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.233195 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.233335 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.233377 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:32.233453 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.230719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46271 (local address 127.0.186.193:46483)
0806 22:40:32.230908 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.230916 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.230972 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.231122 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.231123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.231125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:32.231196 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.231197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.231847 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.231848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.232672 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.232675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.233201 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.233202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.233264 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.233267 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.233311 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.233350 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":23,"spinlock_wait_cycles":768,"thread_start_us":142,"threads_started":1}
I20260806 22:40:32.233459 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.230710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:45713 (local address 127.0.186.193:37299)
0806 22:40:32.230872 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.230883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.230952 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.231080 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.231081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.231083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:32.231157 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.231159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.231807 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.231808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.232670 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.232673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.233215 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.233217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.233271 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.233274 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.233319 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.233359 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":30,"thread_start_us":68,"threads_started":1}
I20260806 22:40:32.233726 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fdf08ff3fc430a95e012a259717066" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7" is_pre_election: true
I20260806 22:40:32.233808 13129 raft_consensus.cc:2454] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 0.
I20260806 22:40:32.233996 12606 leader_election.cc:302] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, a4fa579b2b59437ea6b76e3baaa289b7; no voters:
I20260806 22:40:32.233978 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fdf08ff3fc430a95e012a259717066" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49" is_pre_election: true
I20260806 22:40:32.234129 12913 raft_consensus.cc:2454] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 0.
I20260806 22:40:32.234138 13248 raft_consensus.cc:2781] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:32.234179 13248 raft_consensus.cc:482] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:32.234195 13248 raft_consensus.cc:3037] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.234886 13248 raft_consensus.cc:504] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.235010 13248 leader_election.cc:288] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 election: Requested vote from peers c9135210befd499499300995f1386a49 (127.0.186.194:45713), 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271)
I20260806 22:40:32.235190 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fdf08ff3fc430a95e012a259717066" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49"
I20260806 22:40:32.235183 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fdf08ff3fc430a95e012a259717066" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7"
I20260806 22:40:32.235283 12913 raft_consensus.cc:3037] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.235283 13129 raft_consensus.cc:3037] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.236040 13129 raft_consensus.cc:2454] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 1.
I20260806 22:40:32.236078 12913 raft_consensus.cc:2454] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 1.
I20260806 22:40:32.236238 12609 leader_election.cc:302] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, a4fa579b2b59437ea6b76e3baaa289b7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:32.236358 13248 raft_consensus.cc:2781] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:32.236500 13248 raft_consensus.cc:686] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Running, Role: LEADER
I20260806 22:40:32.236574 13248 consensus_queue.cc:237] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.237231 12477 catalog_manager.cc:5495] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056031, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.254@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:32.302634 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.295997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.0.186.254:40381)
0806 22:40:32.296127 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.296129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.296138 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:32.296184 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.296185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.296193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.296250 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:32.296728 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.297229 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.297922 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.298126 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.298174 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.300230 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:32.300232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:32.300232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:32.300238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:32.301559 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.301905 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.301905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.301906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.301961 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.302193 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.302194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.302194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.302312 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:32.302377 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.302487 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.302561 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260806 22:40:32.302742 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.295948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:40381 (local address 127.0.0.1:44136)
0806 22:40:32.296056 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.296068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.296139 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.296288 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.296289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.296615 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:32.296665 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.296668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.297307 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.297309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.297842 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.297843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.298179 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.298179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.298216 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.298707 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:32.298712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:32.300154 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:32.301624 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:32.301631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:32.301633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:32.301824 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:32.302037 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:32.302038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:32.302039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:32.302104 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:32.302369 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:32.302376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:32.302420 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.302465 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.302625 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":62,"thread_start_us":44,"threads_started":1}
I20260806 22:40:32.304668 12477 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:44136:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:40:32.304898 12477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:32.306335 12882 tablet_service.cc:1467] Processing CreateTablet for tablet 0aabd255a23c4872b67556c1f94e8455 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be11a3ec4fc94859bc80423b1e5515a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:32.306329 12691 tablet_service.cc:1467] Processing CreateTablet for tablet 0aabd255a23c4872b67556c1f94e8455 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be11a3ec4fc94859bc80423b1e5515a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:32.306339 13092 tablet_service.cc:1467] Processing CreateTablet for tablet 0aabd255a23c4872b67556c1f94e8455 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be11a3ec4fc94859bc80423b1e5515a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:40:32.306488 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aabd255a23c4872b67556c1f94e8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.306488 12691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aabd255a23c4872b67556c1f94e8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.306488 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aabd255a23c4872b67556c1f94e8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.307688 13238 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: Bootstrap starting.
I20260806 22:40:32.307986 13241 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: Bootstrap starting.
I20260806 22:40:32.307996 13239 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: Bootstrap starting.
I20260806 22:40:32.308043 13238 tablet_bootstrap.cc:654] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.308329 13241 tablet_bootstrap.cc:654] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.308391 13239 tablet_bootstrap.cc:654] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.308610 13238 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: No bootstrap required, opened a new log
I20260806 22:40:32.308660 13238 ts_tablet_manager.cc:1397] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:32.308791 13238 raft_consensus.cc:348] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.308873 13238 raft_consensus.cc:374] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.308909 13238 raft_consensus.cc:729] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9135210befd499499300995f1386a49, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.308964 13238 consensus_queue.cc:260] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.308984 13239 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: No bootstrap required, opened a new log
I20260806 22:40:32.309028 13239 ts_tablet_manager.cc:1397] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:32.309065 13238 ts_tablet_manager.cc:1428] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:32.309132 13241 tablet_bootstrap.cc:492] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: No bootstrap required, opened a new log
I20260806 22:40:32.309177 13241 ts_tablet_manager.cc:1397] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:32.309145 13249 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.309162 13239 raft_consensus.cc:348] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.309226 13239 raft_consensus.cc:374] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.309211 13249 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.309247 13239 raft_consensus.cc:729] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.309288 13239 consensus_queue.cc:260] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.309314 13241 raft_consensus.cc:348] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.309373 13239 ts_tablet_manager.cc:1428] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:32.309377 13241 raft_consensus.cc:374] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.309410 13241 raft_consensus.cc:729] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.309449 13241 consensus_queue.cc:260] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.309441 13248 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.309513 13248 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.309545 13241 ts_tablet_manager.cc:1428] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:32.309646 13255 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.309724 13255 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.453672 13248 raft_consensus.cc:482] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:32.453774 13248 raft_consensus.cc:504] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.453908 13248 leader_election.cc:288] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.454128 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aabd255a23c4872b67556c1f94e8455" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49" is_pre_election: true
I20260806 22:40:32.454185 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aabd255a23c4872b67556c1f94e8455" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7" is_pre_election: true
I20260806 22:40:32.454263 13129 raft_consensus.cc:2454] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 0.
I20260806 22:40:32.454259 12913 raft_consensus.cc:2454] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 0.
I20260806 22:40:32.454411 12609 leader_election.cc:302] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fa579b2b59437ea6b76e3baaa289b7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:32.454520 13248 raft_consensus.cc:2781] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:32.454564 13248 raft_consensus.cc:482] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:32.454579 13248 raft_consensus.cc:3037] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.455229 13248 raft_consensus.cc:504] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.455394 13248 leader_election.cc:288] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 election: Requested vote from peers 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.455574 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aabd255a23c4872b67556c1f94e8455" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49"
I20260806 22:40:32.455574 13129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aabd255a23c4872b67556c1f94e8455" candidate_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a4526fa24544f3aae67dffb540df3f7"
I20260806 22:40:32.455672 12913 raft_consensus.cc:3037] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.455672 13129 raft_consensus.cc:3037] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.456457 12913 raft_consensus.cc:2454] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 1.
I20260806 22:40:32.456457 13129 raft_consensus.cc:2454] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fa579b2b59437ea6b76e3baaa289b7 in term 1.
I20260806 22:40:32.456678 12609 leader_election.cc:302] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fa579b2b59437ea6b76e3baaa289b7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:32.456792 13248 raft_consensus.cc:2781] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:32.456835 13248 raft_consensus.cc:686] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Running, Role: LEADER
I20260806 22:40:32.456884 13248 consensus_queue.cc:237] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } }
I20260806 22:40:32.457147 13297 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: TxnStatusTablet state changed. Reason: New leader a4fa579b2b59437ea6b76e3baaa289b7. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.457233 13297 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:32.457443 13320 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:40:32.457475 12478 catalog_manager.cc:5495] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193). New cstate: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:32.457579 13320 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:40:32.492758 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.489877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.0.186.193:40883)
0806 22:40:32.489970 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.489972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.489979 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:32.490013 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.490013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.490045 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.490131 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.490251 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.490780 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.491925 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.492131 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.492188 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.492420 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.492523 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.492601 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.492658 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.492679 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":47,"threads_started":1}
I20260806 22:40:32.492794 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.489808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.0.1:59464)
0806 22:40:32.489893 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.489904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.489945 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.490126 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.490127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.490128 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:32.490187 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.490190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.490856 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.490857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.491819 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.491822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.492199 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.492200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.492268 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.492287 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.492598 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.492604 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.492654 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.492708 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:40:32.494064 13129 raft_consensus.cc:1261] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Refusing update from remote peer a4fa579b2b59437ea6b76e3baaa289b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.494338 12913 raft_consensus.cc:1261] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Refusing update from remote peer a4fa579b2b59437ea6b76e3baaa289b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.494334 13248 consensus_queue.cc:1035] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.494596 13248 consensus_queue.cc:1035] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.494906 13255 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: TxnStatusTablet state changed. Reason: New leader a4fa579b2b59437ea6b76e3baaa289b7. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.494998 13255 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.495298 13249 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: TxnStatusTablet state changed. Reason: New leader a4fa579b2b59437ea6b76e3baaa289b7. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.495385 13249 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.496151 13297 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.496241 13297 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:32.496344 13297 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.496402 13297 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P a4fa579b2b59437ea6b76e3baaa289b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:40:32.496522 13249 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.496596 13249 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P c9135210befd499499300995f1386a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.496711 13255 tablet_replica.cc:439] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } }
I20260806 22:40:32.496793 13255 tablet_replica.cc:442] T 0aabd255a23c4872b67556c1f94e8455 P 1a4526fa24544f3aae67dffb540df3f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:40:32.500948 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.498767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.0.186.193:40883)
0806 22:40:32.498807 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.498809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.498870 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:40:32.498931 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.498932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.498942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.498985 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.499108 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.499649 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.500361 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.500559 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.500611 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.500746 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.500777 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.500818 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.500867 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.500887 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:32.506028 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.503410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:44071 (local address 127.0.186.193:40883)
0806 22:40:32.503457 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.503459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.503585 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:40:32.503645 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.503646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.503654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.503695 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.503812 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.504243 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.505364 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.505595 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.505644 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.505816 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.505838 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.505866 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.505917 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.505932 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:32.506147 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.503364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.186.193:44071)
0806 22:40:32.503513 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.503523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.503591 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.503700 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.503700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.503702 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:32.503765 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.503767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.504304 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.504305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.505291 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.505294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.505661 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.505663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.505729 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.505745 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.505869 (+ 124us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.505870 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.505915 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.505955 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260806 22:40:32.508378 13129 raft_consensus.cc:1261] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Refusing update from remote peer a4fa579b2b59437ea6b76e3baaa289b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.508378 12913 raft_consensus.cc:1261] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Refusing update from remote peer a4fa579b2b59437ea6b76e3baaa289b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.508566 13297 consensus_queue.cc:1035] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.508667 13248 consensus_queue.cc:1035] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.520042 12476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.520138 12476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.520766 12476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Sending DeleteTablet for 3 replicas of tablet 52fdf08ff3fc430a95e012a259717066
I20260806 22:40:32.521080 13092 tablet_service.cc:1514] Processing DeleteTablet for tablet 52fdf08ff3fc430a95e012a259717066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:53862
I20260806 22:40:32.521077 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet 52fdf08ff3fc430a95e012a259717066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49232
I20260806 22:40:32.521548 13352 tablet_replica.cc:330] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: stopping tablet replica
I20260806 22:40:32.521608 13352 raft_consensus.cc:2229] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.521639 13352 raft_consensus.cc:2258] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.521899 13352 ts_tablet_manager.cc:1905] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.523252 12688 tablet_service.cc:1514] Processing DeleteTablet for tablet 52fdf08ff3fc430a95e012a259717066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260806 22:40:32.523628 13350 tablet_replica.cc:330] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: stopping tablet replica
I20260806 22:40:32.523695 13350 raft_consensus.cc:2229] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.523730 13350 raft_consensus.cc:2258] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.523720 13352 ts_tablet_manager.cc:1918] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.523768 13352 log.cc:1196] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/52fdf08ff3fc430a95e012a259717066
I20260806 22:40:32.523933 13352 ts_tablet_manager.cc:1939] T 52fdf08ff3fc430a95e012a259717066 P 1a4526fa24544f3aae67dffb540df3f7: Deleting consensus metadata
I20260806 22:40:32.524397 12457 catalog_manager.cc:4841] TS 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271): tablet 52fdf08ff3fc430a95e012a259717066 (table test-table [id=942179defef349a49af1513749882c12]) successfully deleted
I20260806 22:40:32.524606 13354 tablet_replica.cc:330] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: stopping tablet replica
I20260806 22:40:32.524626 13350 ts_tablet_manager.cc:1905] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.524672 13354 raft_consensus.cc:2229] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:32.524747 13354 raft_consensus.cc:2258] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.524991 13354 ts_tablet_manager.cc:1905] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.526068 13350 ts_tablet_manager.cc:1918] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.526127 13350 log.cc:1196] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/52fdf08ff3fc430a95e012a259717066
I20260806 22:40:32.526333 13350 ts_tablet_manager.cc:1939] T 52fdf08ff3fc430a95e012a259717066 P c9135210befd499499300995f1386a49: Deleting consensus metadata
I20260806 22:40:32.526618 12459 catalog_manager.cc:4841] TS c9135210befd499499300995f1386a49 (127.0.186.194:45713): tablet 52fdf08ff3fc430a95e012a259717066 (table test-table [id=942179defef349a49af1513749882c12]) successfully deleted
I20260806 22:40:32.526743 13354 ts_tablet_manager.cc:1918] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.526809 13354 log.cc:1196] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/52fdf08ff3fc430a95e012a259717066
I20260806 22:40:32.527034 13354 ts_tablet_manager.cc:1939] T 52fdf08ff3fc430a95e012a259717066 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting consensus metadata
I20260806 22:40:32.527339 12459 catalog_manager.cc:4841] TS a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883): tablet 52fdf08ff3fc430a95e012a259717066 (table test-table [id=942179defef349a49af1513749882c12]) successfully deleted
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[12414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:40:32.555102 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:32.549196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.0.186.254:40381)
0806 22:40:32.549270 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.549272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.549288 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:40:32.549354 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.549356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.549368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.549444 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:32.550071 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.550897 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.551706 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.551999 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.552096 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.555021 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44140: BlockingRecv error: recv got EOF from 127.0.0.1:44140 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[12414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM
I20260806 22:40:32.563205 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.555862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.0.186.254:40381)
0806 22:40:32.555901 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.555903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.555955 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:40:32.556018 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.556018 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.556036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.556084 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:32.556521 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.557244 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.558001 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.558301 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.558376 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.560658 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:32.560659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:32.560660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:32.560668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:32.562161 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.562494 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.562496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.562496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.562563 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.562718 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.562719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.562719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.562852 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:32.562973 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.563017 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.563108 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:32.564766 12476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:32.564963 12476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:32.566434 12688 tablet_service.cc:1467] Processing CreateTablet for tablet d07fcef255104243b27f3643a963a0b5 (DEFAULT_TABLE table=test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.566428 13092 tablet_service.cc:1467] Processing CreateTablet for tablet d07fcef255104243b27f3643a963a0b5 (DEFAULT_TABLE table=test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.566571 12688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d07fcef255104243b27f3643a963a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.566589 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d07fcef255104243b27f3643a963a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.567420 12882 tablet_service.cc:1467] Processing CreateTablet for tablet d07fcef255104243b27f3643a963a0b5 (DEFAULT_TABLE table=test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.567541 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d07fcef255104243b27f3643a963a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.567976 13241 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Bootstrap starting.
I20260806 22:40:32.568107 13239 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Bootstrap starting.
I20260806 22:40:32.568380 13241 tablet_bootstrap.cc:654] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.568490 13239 tablet_bootstrap.cc:654] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.568845 13238 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Bootstrap starting.
I20260806 22:40:32.569193 13238 tablet_bootstrap.cc:654] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.569329 13241 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: No bootstrap required, opened a new log
I20260806 22:40:32.569380 13241 ts_tablet_manager.cc:1397] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:32.569537 13241 raft_consensus.cc:348] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.569618 13241 raft_consensus.cc:374] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.569623 13239 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: No bootstrap required, opened a new log
I20260806 22:40:32.569639 13241 raft_consensus.cc:729] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.569664 13239 ts_tablet_manager.cc:1397] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:32.569682 13241 consensus_queue.cc:260] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.569772 13241 ts_tablet_manager.cc:1428] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:32.569808 13239 raft_consensus.cc:348] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.569877 13239 raft_consensus.cc:374] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.569900 13239 raft_consensus.cc:729] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.569938 13239 consensus_queue.cc:260] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.570022 13239 ts_tablet_manager.cc:1428] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:32.570727 13238 tablet_bootstrap.cc:492] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: No bootstrap required, opened a new log
I20260806 22:40:32.570782 13238 ts_tablet_manager.cc:1397] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:32.570911 13238 raft_consensus.cc:348] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.570979 13238 raft_consensus.cc:374] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.571004 13238 raft_consensus.cc:729] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9135210befd499499300995f1386a49, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.571062 13238 consensus_queue.cc:260] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.571161 13238 ts_tablet_manager.cc:1428] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:32.699329 13255 raft_consensus.cc:482] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:32.699432 13255 raft_consensus.cc:504] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.699775 13255 leader_election.cc:288] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.703517 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.700086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:54759 (local address 127.0.186.193:40883)
0806 22:40:32.700140 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.700142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.700314 (+ 172us) server_negotiation.cc:407] Connection header received
0806 22:40:32.700429 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.700430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.700440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.700470 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:32.700731 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.701450 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.702743 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.703110 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.703171 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.703181 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.703398 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.703441 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:40:32.703637 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.700382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:48093 (local address 127.0.186.194:45713)
0806 22:40:32.700425 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.700426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.700552 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:40:32.700593 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.700594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.700605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.700660 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:32.700799 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.701270 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.702713 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.703324 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.703386 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.703395 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.703526 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.703565 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:32.703856 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.700303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:45713 (local address 127.0.186.195:48093)
0806 22:40:32.700472 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.700482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.700538 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.700670 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.700670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.700672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:32.700737 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.700738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.701352 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.701353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.702616 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.702619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.703400 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.703401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.703462 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.703466 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.703512 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.703552 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260806 22:40:32.704028 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.700046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.186.195:54759)
0806 22:40:32.700237 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.700248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.700347 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.700542 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.700544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.700546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:32.700635 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.700637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.701550 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.701551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.702628 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.702630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.703207 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.703209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.703308 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.703313 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.703941 (+ 628us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.703982 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
I20260806 22:40:32.704237 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d07fcef255104243b27f3643a963a0b5" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49" is_pre_election: true
I20260806 22:40:32.704238 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d07fcef255104243b27f3643a963a0b5" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" is_pre_election: true
I20260806 22:40:32.704337 12913 raft_consensus.cc:2454] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 0.
I20260806 22:40:32.704340 12718 raft_consensus.cc:2454] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 0.
I20260806 22:40:32.704484 13018 leader_election.cc:302] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:32.704623 13255 raft_consensus.cc:2781] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:32.704671 13255 raft_consensus.cc:482] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:32.704689 13255 raft_consensus.cc:3037] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.705607 13255 raft_consensus.cc:504] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.705758 13255 leader_election.cc:288] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 election: Requested vote from peers a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.705961 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d07fcef255104243b27f3643a963a0b5" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
I20260806 22:40:32.705965 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d07fcef255104243b27f3643a963a0b5" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49"
I20260806 22:40:32.706043 12718 raft_consensus.cc:3037] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.706043 12913 raft_consensus.cc:3037] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.706799 12718 raft_consensus.cc:2454] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 1.
I20260806 22:40:32.706800 12913 raft_consensus.cc:2454] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 1.
I20260806 22:40:32.706954 13018 leader_election.cc:302] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, a4fa579b2b59437ea6b76e3baaa289b7; no voters:
I20260806 22:40:32.707072 13255 raft_consensus.cc:2781] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:32.707249 13255 raft_consensus.cc:686] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Running, Role: LEADER
I20260806 22:40:32.707386 13255 consensus_queue.cc:237] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.708127 12476 catalog_manager.cc:5495] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "1a4526fa24544f3aae67dffb540df3f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:32.716784 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.714792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.0.186.195:46271)
0806 22:40:32.714839 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.714841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.714848 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:32.714878 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.714879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.714895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.714953 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.715052 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.715556 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.716215 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.716382 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.716422 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.716515 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.716624 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.716672 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.716704 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.716718 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:32.717684 12913 raft_consensus.cc:1261] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Refusing update from remote peer 1a4526fa24544f3aae67dffb540df3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.717702 12718 raft_consensus.cc:1261] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Refusing update from remote peer 1a4526fa24544f3aae67dffb540df3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.717907 13255 consensus_queue.cc:1035] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.718041 13378 consensus_queue.cc:1035] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.721192 12478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.721278 12478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.721853 12478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Sending DeleteTablet for 3 replicas of tablet d07fcef255104243b27f3643a963a0b5
I20260806 22:40:32.722150 13092 tablet_service.cc:1514] Processing DeleteTablet for tablet d07fcef255104243b27f3643a963a0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:53862
I20260806 22:40:32.722239 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet d07fcef255104243b27f3643a963a0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49232
I20260806 22:40:32.722334 13352 tablet_replica.cc:330] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: stopping tablet replica
I20260806 22:40:32.722407 13352 raft_consensus.cc:2229] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:32.722352 12478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:32.722483 12478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:32.722488 13352 raft_consensus.cc:2258] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.722564 13350 tablet_replica.cc:330] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: stopping tablet replica
I20260806 22:40:32.722630 13350 raft_consensus.cc:2229] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.722674 13350 raft_consensus.cc:2258] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.722872 12688 tablet_service.cc:1514] Processing DeleteTablet for tablet d07fcef255104243b27f3643a963a0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260806 22:40:32.722981 13354 tablet_replica.cc:330] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: stopping tablet replica
I20260806 22:40:32.723067 13354 raft_consensus.cc:2229] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.723129 13354 raft_consensus.cc:2258] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.723251 13352 ts_tablet_manager.cc:1905] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.723451 13350 ts_tablet_manager.cc:1905] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.724877 13352 ts_tablet_manager.cc:1918] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:32.724931 13352 log.cc:1196] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/d07fcef255104243b27f3643a963a0b5
I20260806 22:40:32.724984 13350 ts_tablet_manager.cc:1918] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:32.725037 13350 log.cc:1196] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/d07fcef255104243b27f3643a963a0b5
I20260806 22:40:32.725015 13092 tablet_service.cc:1467] Processing CreateTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 (DEFAULT_TABLE table=test-table [id=f9c1b6751616492988624f74172c4ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.725116 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780a725eaf2448e9849dcb54a212fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.725153 13352 ts_tablet_manager.cc:1939] T d07fcef255104243b27f3643a963a0b5 P 1a4526fa24544f3aae67dffb540df3f7: Deleting consensus metadata
I20260806 22:40:32.725242 13350 ts_tablet_manager.cc:1939] T d07fcef255104243b27f3643a963a0b5 P c9135210befd499499300995f1386a49: Deleting consensus metadata
I20260806 22:40:32.725574 13354 ts_tablet_manager.cc:1905] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.726075 13241 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Bootstrap starting.
I20260806 22:40:32.726105 12882 tablet_service.cc:1467] Processing CreateTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 (DEFAULT_TABLE table=test-table [id=f9c1b6751616492988624f74172c4ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.726189 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780a725eaf2448e9849dcb54a212fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.726460 13241 tablet_bootstrap.cc:654] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.727010 13354 ts_tablet_manager.cc:1918] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:32.727063 13354 log.cc:1196] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/d07fcef255104243b27f3643a963a0b5
I20260806 22:40:32.727241 13238 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Bootstrap starting.
I20260806 22:40:32.727303 13354 ts_tablet_manager.cc:1939] T d07fcef255104243b27f3643a963a0b5 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting consensus metadata
I20260806 22:40:32.727373 12688 tablet_service.cc:1467] Processing CreateTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 (DEFAULT_TABLE table=test-table [id=f9c1b6751616492988624f74172c4ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:32.727471 12688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780a725eaf2448e9849dcb54a212fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.727553 13238 tablet_bootstrap.cc:654] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.728310 13238 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: No bootstrap required, opened a new log
I20260806 22:40:32.728364 13238 ts_tablet_manager.cc:1397] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:32.728466 13238 raft_consensus.cc:348] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.728545 13238 raft_consensus.cc:374] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.728571 13238 raft_consensus.cc:729] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9135210befd499499300995f1386a49, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.728613 13238 consensus_queue.cc:260] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.728708 13238 ts_tablet_manager.cc:1428] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:32.728703 13239 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Bootstrap starting.
I20260806 22:40:32.728849 12457 catalog_manager.cc:4841] TS 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271): tablet d07fcef255104243b27f3643a963a0b5 (table test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]) successfully deleted
I20260806 22:40:32.728978 13239 tablet_bootstrap.cc:654] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:32.729059 12459 catalog_manager.cc:4841] TS a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883): tablet d07fcef255104243b27f3643a963a0b5 (table test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]) successfully deleted
I20260806 22:40:32.729135 12459 catalog_manager.cc:4841] TS c9135210befd499499300995f1386a49 (127.0.186.194:45713): tablet d07fcef255104243b27f3643a963a0b5 (table test-table [id=cdbe8ea5a9744c2eac7f7d96ad9b7114]) successfully deleted
I20260806 22:40:32.729460 13239 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: No bootstrap required, opened a new log
I20260806 22:40:32.729511 13239 ts_tablet_manager.cc:1397] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:32.729601 13241 tablet_bootstrap.cc:492] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: No bootstrap required, opened a new log
I20260806 22:40:32.729648 13241 ts_tablet_manager.cc:1397] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260806 22:40:32.729629 13239 raft_consensus.cc:348] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.729697 13239 raft_consensus.cc:374] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.729722 13239 raft_consensus.cc:729] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fa579b2b59437ea6b76e3baaa289b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.729764 13239 consensus_queue.cc:260] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.729789 13241 raft_consensus.cc:348] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.729853 13239 ts_tablet_manager.cc:1428] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:32.729856 13241 raft_consensus.cc:374] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:32.729882 13241 raft_consensus.cc:729] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Initialized, Role: FOLLOWER
I20260806 22:40:32.729924 13241 consensus_queue.cc:260] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.730006 13241 ts_tablet_manager.cc:1428] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:32.770511 13389 raft_consensus.cc:482] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:32.770640 13389 raft_consensus.cc:504] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.770835 13389 leader_election.cc:288] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.771100 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780a725eaf2448e9849dcb54a212fe4" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" is_pre_election: true
I20260806 22:40:32.771100 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780a725eaf2448e9849dcb54a212fe4" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49" is_pre_election: true
I20260806 22:40:32.771224 12718 raft_consensus.cc:2454] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 0.
I20260806 22:40:32.771224 12913 raft_consensus.cc:2454] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 0.
I20260806 22:40:32.771416 13018 leader_election.cc:302] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, a4fa579b2b59437ea6b76e3baaa289b7; no voters:
I20260806 22:40:32.771551 13389 raft_consensus.cc:2781] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:32.771617 13389 raft_consensus.cc:482] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:32.771632 13389 raft_consensus.cc:3037] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.772452 13389 raft_consensus.cc:504] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.772562 13389 leader_election.cc:288] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 election: Requested vote from peers a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883), c9135210befd499499300995f1386a49 (127.0.186.194:45713)
I20260806 22:40:32.772708 12718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780a725eaf2448e9849dcb54a212fe4" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fa579b2b59437ea6b76e3baaa289b7"
I20260806 22:40:32.772740 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780a725eaf2448e9849dcb54a212fe4" candidate_uuid: "1a4526fa24544f3aae67dffb540df3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9135210befd499499300995f1386a49"
I20260806 22:40:32.772786 12718 raft_consensus.cc:3037] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.772797 12913 raft_consensus.cc:3037] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:32.773566 12913 raft_consensus.cc:2454] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 1.
I20260806 22:40:32.773566 12718 raft_consensus.cc:2454] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a4526fa24544f3aae67dffb540df3f7 in term 1.
I20260806 22:40:32.773727 13018 leader_election.cc:302] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a4526fa24544f3aae67dffb540df3f7, c9135210befd499499300995f1386a49; no voters:
I20260806 22:40:32.773854 13389 raft_consensus.cc:2781] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:32.773922 13389 raft_consensus.cc:686] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 1a4526fa24544f3aae67dffb540df3f7, State: Running, Role: LEADER
I20260806 22:40:32.773977 13389 consensus_queue.cc:237] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } }
I20260806 22:40:32.774554 12478 catalog_manager.cc:5495] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "1a4526fa24544f3aae67dffb540df3f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a4526fa24544f3aae67dffb540df3f7" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:32.792048 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.789660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:54499 (local address 127.0.186.193:40883)
0806 22:40:32.789705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.789707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.789847 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:40:32.789912 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.789912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.789925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.789973 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.790115 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.790600 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.791412 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.791605 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.791660 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.791814 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.791848 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.791889 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.791938 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.791971 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:32.792085 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.789586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40883 (local address 127.0.186.195:54499)
0806 22:40:32.789781 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.789791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.789850 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.789985 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.789986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.789987 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:32.790057 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.790058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.790670 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.790671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.791339 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.791342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.791664 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.791665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.791731 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.791747 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.791894 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.791896 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.791936 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.791980 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260806 22:40:32.794318 12718 raft_consensus.cc:1261] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Refusing update from remote peer 1a4526fa24544f3aae67dffb540df3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.794508 12913 raft_consensus.cc:1261] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Refusing update from remote peer 1a4526fa24544f3aae67dffb540df3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:32.794559 13255 consensus_queue.cc:1035] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fa579b2b59437ea6b76e3baaa289b7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.794787 13255 consensus_queue.cc:1035] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9135210befd499499300995f1386a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:32.801712 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.799537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46271 (local address 127.0.186.193:59907)
0806 22:40:32.799579 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.799590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.799639 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.799757 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.799758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.799759 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:40:32.799802 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.799803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.800391 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.800392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.800937 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.800938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.801316 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.801318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.801361 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.801367 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.801520 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.801523 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.801576 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.801634 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:40:32.801939 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.799615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:59907 (local address 127.0.186.195:46271)
0806 22:40:32.799655 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.799657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.799666 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:32.799689 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.799689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.799699 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.799752 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:32.799848 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.800319 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.801012 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.801223 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.801283 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.801430 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:32.801465 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:32.801513 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.801824 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.801849 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:40:32.808385 12478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.808521 12478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:32.809227 12478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f25d80326bd4810b8256da83d53a608: Sending DeleteTablet for 3 replicas of tablet 5780a725eaf2448e9849dcb54a212fe4
I20260806 22:40:32.809491 13092 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:53862
I20260806 22:40:32.809584 12688 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260806 22:40:32.809729 13354 tablet_replica.cc:330] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: stopping tablet replica
I20260806 22:40:32.809795 13354 raft_consensus.cc:2229] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.809834 13354 raft_consensus.cc:2258] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.810026 13352 tablet_replica.cc:330] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: stopping tablet replica
I20260806 22:40:32.810025 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780a725eaf2448e9849dcb54a212fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:32 UTC) from {username='oryx', principal='oryx/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49232
I20260806 22:40:32.810091 13352 raft_consensus.cc:2229] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:32.810104 13350 tablet_replica.cc:330] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: stopping tablet replica
I20260806 22:40:32.810145 13352 raft_consensus.cc:2258] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.810145 13350 raft_consensus.cc:2229] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:32.810178 13350 raft_consensus.cc:2258] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:32.810217 13354 ts_tablet_manager.cc:1905] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.810417 13352 ts_tablet_manager.cc:1905] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.810638 13350 ts_tablet_manager.cc:1905] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:32.811952 13354 ts_tablet_manager.cc:1918] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.811965 13350 ts_tablet_manager.cc:1918] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.811971 13352 ts_tablet_manager.cc:1918] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:40:32.812007 13354 log.cc:1196] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/5780a725eaf2448e9849dcb54a212fe4
I20260806 22:40:32.812016 13350 log.cc:1196] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/5780a725eaf2448e9849dcb54a212fe4
I20260806 22:40:32.812019 13352 log.cc:1196] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/5780a725eaf2448e9849dcb54a212fe4
I20260806 22:40:32.812227 13350 ts_tablet_manager.cc:1939] T 5780a725eaf2448e9849dcb54a212fe4 P c9135210befd499499300995f1386a49: Deleting consensus metadata
I20260806 22:40:32.812227 13352 ts_tablet_manager.cc:1939] T 5780a725eaf2448e9849dcb54a212fe4 P 1a4526fa24544f3aae67dffb540df3f7: Deleting consensus metadata
I20260806 22:40:32.812234 13354 ts_tablet_manager.cc:1939] T 5780a725eaf2448e9849dcb54a212fe4 P a4fa579b2b59437ea6b76e3baaa289b7: Deleting consensus metadata
I20260806 22:40:32.812558 12459 catalog_manager.cc:4841] TS a4fa579b2b59437ea6b76e3baaa289b7 (127.0.186.193:40883): tablet 5780a725eaf2448e9849dcb54a212fe4 (table test-table [id=f9c1b6751616492988624f74172c4ade]) successfully deleted
I20260806 22:40:32.812657 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 12584
I20260806 22:40:32.823155 12459 catalog_manager.cc:4841] TS c9135210befd499499300995f1386a49 (127.0.186.194:45713): tablet 5780a725eaf2448e9849dcb54a212fe4 (table test-table [id=f9c1b6751616492988624f74172c4ade]) successfully deleted
I20260806 22:40:32.831740 12457 catalog_manager.cc:4841] TS 1a4526fa24544f3aae67dffb540df3f7 (127.0.186.195:46271): tablet 5780a725eaf2448e9849dcb54a212fe4 (table test-table [id=f9c1b6751616492988624f74172c4ade]) successfully deleted
I20260806 22:40:32.832095 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 12789
I20260806 22:40:32.849692 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 12991
I20260806 22:40:32.865692 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 12428
2026-08-06T22:40:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3015 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[13410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[13410](info): set up 2 sockets
Aug 06 22:40:32 dist-test-slave-gjrz krb5kdc[13410](info): commencing operation
krb5kdc: starting...
W20260806 22:40:34.008658 747 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:34 dist-test-slave-gjrz krb5kdc[13410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:34.130684 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:41961
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:34.212025 13429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:34.212234 13429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:34.212255 13429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:34.213346 13429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:34.213377 13429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:34.213390 13429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:34.213402 13429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:34.213418 13429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:34.214886 13429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:34.215108 13429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:34.215303 13429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:34.219831 13429 server_base.cc:1034] running on GCE node
W20260806 22:40:34.219774 13439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:34.219791 13435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:34.219796 13437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:34.220274 13429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:34.220575 13429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:34.221769 13429 hybrid_clock.cc:648] HybridClock initialized: now 1786056034221745 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:40:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1359 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:34 dist-test-slave-gjrz krb5kdc[13444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:34 dist-test-slave-gjrz krb5kdc[13444](info): set up 2 sockets
Aug 06 22:40:34 dist-test-slave-gjrz krb5kdc[13444](info): commencing operation
krb5kdc: starting...
W20260806 22:40:35.362330 747 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:35.477815 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36759
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:36759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:35.561730 13469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:35.561967 13469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:35.562008 13469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:35.563181 13469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:35.563222 13469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:35.563241 13469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:35.563256 13469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:35.563303 13469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:35.564862 13469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:36759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:35.565122 13469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:35.565332 13469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:35.570150 13485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:35.570155 13482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:35.570370 13469 server_base.cc:1034] running on GCE node
W20260806 22:40:35.570168 13475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:35.570674 13469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:35.570957 13469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:35.572139 13469 hybrid_clock.cc:648] HybridClock initialized: now 1786056035572124 us; error 31 us; skew 500 ppm
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:35.579842 13469 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:35.580289 13469 webserver.cc:460] Webserver started at http://127.0.186.254:41775/ using document root <none> and password file <none>
I20260806 22:40:35.580490 13469 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:35.580538 13469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:35.580674 13469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:35.581722 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "7866d9ab19df49c59ab3e8e53989ca81"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "a02bffac27b24e9c9c28f5de716a7463"
server_key_iv: "7c4381df32ac186c91916e659d6f0731"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.582072 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "7866d9ab19df49c59ab3e8e53989ca81"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "a02bffac27b24e9c9c28f5de716a7463"
server_key_iv: "7c4381df32ac186c91916e659d6f0731"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.583424 13469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:40:35.584223 13499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.584383 13469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:35.584468 13469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "7866d9ab19df49c59ab3e8e53989ca81"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "a02bffac27b24e9c9c28f5de716a7463"
server_key_iv: "7c4381df32ac186c91916e659d6f0731"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.584537 13469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:35.596031 13469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:35.597915 13469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:35.598052 13469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:35.601970 13469 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:36759
I20260806 22:40:35.602021 13566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:36759 every 8 connection(s)
I20260806 22:40:35.602486 13469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:35.602984 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 13469
I20260806 22:40:35.603093 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:35.603305 13567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:35.603318 747 external_mini_cluster.cc:1428] Setting key 8a01d5860d9864b6b602dff45b405e49
I20260806 22:40:35.606470 13567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Bootstrap starting.
I20260806 22:40:35.607203 13567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:35.607504 13567 log.cc:824] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:35.608299 13567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: No bootstrap required, opened a new log
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:35.610539 13567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:35.610728 13567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:35.610807 13567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7866d9ab19df49c59ab3e8e53989ca81, State: Initialized, Role: FOLLOWER
I20260806 22:40:35.610963 13567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:35.611048 13567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:35.611095 13567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:35.611138 13567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:35.611729 13567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:35.611833 13567 leader_election.cc:302] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7866d9ab19df49c59ab3e8e53989ca81; no voters:
I20260806 22:40:35.612004 13567 leader_election.cc:288] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:35.612064 13574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:35.612216 13574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 LEADER]: Becoming Leader. State: Replica: 7866d9ab19df49c59ab3e8e53989ca81, State: Running, Role: LEADER
I20260806 22:40:35.612259 13567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:35.612382 13574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:35.612798 13575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7866d9ab19df49c59ab3e8e53989ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } } }
I20260806 22:40:35.612886 13575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:35.613111 13577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7866d9ab19df49c59ab3e8e53989ca81. Latest consensus state: current_term: 1 leader_uuid: "7866d9ab19df49c59ab3e8e53989ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } } }
I20260806 22:40:35.613184 13577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:35.613559 13585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:35.613966 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.603849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.0.186.254:36759)
0806 22:40:35.604025 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.604033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.604058 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:40:35.604461 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.604468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.604515 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.604627 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:35.605383 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.606300 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.607306 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.607793 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.607908 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.610348 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:35.610352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:35.610352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:35.610363 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:35.612101 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.612550 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.612554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.612554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.612664 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.612869 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.612870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.612870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.613200 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:35.613357 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.613545 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.613698 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260806 22:40:35.614159 13585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:35.615598 13585 catalog_manager.cc:1347] Generated new cluster ID: 75c3fa0d23424c4dbadfae59fdc1b144
I20260806 22:40:35.615640 13585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:35.628163 13585 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:35.628713 13585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:35.631880 13585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Generated new TSK 0
I20260806 22:40:35.632077 13585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:35.664449 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:35.770486 13606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:35.770687 13606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:35.770709 13606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:35.771876 13606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:35.771919 13606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:35.771943 13606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:35.773420 13606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:35.773641 13606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:35.773849 13606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:35.778654 13619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:35.778672 13617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:35.778754 13606 server_base.cc:1034] running on GCE node
W20260806 22:40:35.778676 13615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:35.779129 13606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:35.779397 13606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:35.780570 13606 hybrid_clock.cc:648] HybridClock initialized: now 1786056035780557 us; error 30 us; skew 500 ppm
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:35.783332 13606 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:35.783787 13606 webserver.cc:460] Webserver started at http://127.0.186.193:40181/ using document root <none> and password file <none>
I20260806 22:40:35.783979 13606 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:35.784020 13606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:35.784111 13606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:35.785105 13606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "8950c889657a4bbbb401a265367c2cfd"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "bd77a2debb519e12f0c41f1e7563269f"
server_key_iv: "4b473f19bcd22a99aaa3cf29c83fb28c"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.785404 13606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "8950c889657a4bbbb401a265367c2cfd"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "bd77a2debb519e12f0c41f1e7563269f"
server_key_iv: "4b473f19bcd22a99aaa3cf29c83fb28c"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.786706 13606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:40:35.787441 13630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.787602 13606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:35.787693 13606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "8950c889657a4bbbb401a265367c2cfd"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "bd77a2debb519e12f0c41f1e7563269f"
server_key_iv: "4b473f19bcd22a99aaa3cf29c83fb28c"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.787753 13606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:35.812886 13606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:35.815137 13606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:35.815254 13606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:35.815478 13606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:35.815763 13606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:35.815794 13606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.815816 13606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:35.815826 13606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.821206 13606 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:40231
I20260806 22:40:35.821280 13793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:40231 every 8 connection(s)
I20260806 22:40:35.821583 13606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:35.830667 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.822308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54051 (local address 127.0.186.254:36759)
0806 22:40:35.822353 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.822356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.822560 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:40:35.823516 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.823518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.823529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.823592 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:35.824212 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.824748 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.825704 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.825907 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.825988 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.828102 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:35.828104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:35.828105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:35.828112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:35.829363 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.829693 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.829694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.829694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.829742 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.829973 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.829975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.829975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.830101 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:35.830171 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.830518 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.830596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:35.830785 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.822277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:54051)
0806 22:40:35.822448 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:35.822492 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:35.823420 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:35.823664 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:35.823668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:35.823922 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:35.824114 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:35.824117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.824825 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.824826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:35.825621 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:35.825623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.825993 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.825993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:35.826090 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.826538 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:35.826547 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:35.828011 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:35.829428 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:35.829432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:35.829434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:35.829636 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:35.829784 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:35.829784 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:35.829785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:35.829871 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:35.830233 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:35.830242 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:35.830368 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:40:35.830466 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:40:35.830597 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260806 22:40:35.831106 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 13606
I20260806 22:40:35.831215 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:35.831364 13794 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36759
I20260806 22:40:35.831451 747 external_mini_cluster.cc:1428] Setting key 975d88f4917bb438daee35345f490cb5
I20260806 22:40:35.831467 13794 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:35.831664 13794 heartbeater.cc:511] Master 127.0.186.254:36759 requested a full tablet report, sending...
I20260806 22:40:35.832237 13522 ts_manager.cc:194] Registered new tserver with Master: 8950c889657a4bbbb401a265367c2cfd (127.0.186.193:40231)
I20260806 22:40:35.832831 13522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:54051
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:35.857810 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:35.939584 13800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:35.939785 13800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:35.939814 13800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:35.940943 13800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:35.940985 13800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:35.941020 13800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:35.942533 13800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:35.942768 13800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:35.942986 13800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:35.947914 13815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:35.947911 13812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:35.947953 13811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:35.948235 13800 server_base.cc:1034] running on GCE node
I20260806 22:40:35.948443 13800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:35.948683 13800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:35.949831 13800 hybrid_clock.cc:648] HybridClock initialized: now 1786056035949814 us; error 28 us; skew 500 ppm
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:35.952371 13800 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:35.952816 13800 webserver.cc:460] Webserver started at http://127.0.186.194:34131/ using document root <none> and password file <none>
I20260806 22:40:35.953047 13800 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:35.953094 13800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:35.953223 13800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:35.954224 13800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "51d338dcc46048f99a57f8862e1d93eb"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "539a1f0b4c898e4673ca8bf822e8b297"
server_key_iv: "0f9abbf186eadaa8d2667b045d310189"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.954559 13800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "51d338dcc46048f99a57f8862e1d93eb"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "539a1f0b4c898e4673ca8bf822e8b297"
server_key_iv: "0f9abbf186eadaa8d2667b045d310189"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.955858 13800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:35.956580 13826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.956736 13800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:35.956810 13800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "51d338dcc46048f99a57f8862e1d93eb"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "539a1f0b4c898e4673ca8bf822e8b297"
server_key_iv: "0f9abbf186eadaa8d2667b045d310189"
server_key_version: "encryptionkey@0"
I20260806 22:40:35.956876 13800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:35.963678 13800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:35.965622 13800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:35.965754 13800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:35.965962 13800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:35.966248 13800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:35.966279 13800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.966306 13800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:35.966327 13800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:35.971966 13800 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43619
I20260806 22:40:35.972019 13981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43619 every 8 connection(s)
I20260806 22:40:35.972316 13800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:35.973034 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 13800
I20260806 22:40:35.973127 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:35.973309 747 external_mini_cluster.cc:1428] Setting key 79b0352166a3a46c59e0a1d208c298bd
Aug 06 22:40:35 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:35.982677 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.973029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45137 (local address 127.0.186.254:36759)
0806 22:40:35.973098 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.973101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.973253 (+ 152us) server_negotiation.cc:407] Connection header received
0806 22:40:35.974367 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.974369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.974379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.974442 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:35.975065 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.975780 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.977193 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.977388 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.977443 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.979765 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:35.979767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:35.979767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:35.979777 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:35.981242 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.981639 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.981640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.981640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.981708 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:35.982009 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:35.982010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:35.982011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:35.982156 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:35.982230 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.982495 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.982582 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:40:35.982754 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.972973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:45137)
0806 22:40:35.973128 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:35.973169 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:35.974263 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:35.974504 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:35.974508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:35.974769 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:35.974984 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:35.974988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.975880 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.975908 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:35.977091 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:35.977095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.977472 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.977474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:35.977613 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.978101 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:35.978111 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:35.979670 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:35.981338 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:35.981344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:35.981346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:35.981568 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:35.981805 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:35.981806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:35.981807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:35.981937 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:35.982230 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:35.982236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:35.982357 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 22:40:35.982451 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:40:35.982582 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
I20260806 22:40:35.983217 13982 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36759
I20260806 22:40:35.983340 13982 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:35.983538 13982 heartbeater.cc:511] Master 127.0.186.254:36759 requested a full tablet report, sending...
I20260806 22:40:35.983918 13522 ts_manager.cc:194] Registered new tserver with Master: 51d338dcc46048f99a57f8862e1d93eb (127.0.186.194:43619)
I20260806 22:40:35.984282 13522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:45137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:36.000164 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:36.081720 13989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:36.081951 13989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:36.081975 13989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:36.083163 13989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:36.083204 13989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:36.083228 13989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:36.084810 13989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:36.085052 13989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:36.085258 13989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:36.090229 14000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.090252 13998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.090231 14004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.090418 13989 server_base.cc:1034] running on GCE node
I20260806 22:40:36.090744 13989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:36.091024 13989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:36.092171 13989 hybrid_clock.cc:648] HybridClock initialized: now 1786056036092159 us; error 27 us; skew 500 ppm
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:36.094624 13989 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:36.095041 13989 webserver.cc:460] Webserver started at http://127.0.186.195:34035/ using document root <none> and password file <none>
I20260806 22:40:36.095323 13989 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:36.095371 13989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:36.095489 13989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:36.096490 13989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "21b2d70740154f889b47154f7522d23c"
format_stamp: "Formatted at 2026-08-06 22:40:36 on dist-test-slave-gjrz"
server_key: "b27356ea3fa0dfc6b414a726ffc65bde"
server_key_iv: "2d3f7fd1ca36bd33445089b86243dc10"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.096817 13989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "21b2d70740154f889b47154f7522d23c"
format_stamp: "Formatted at 2026-08-06 22:40:36 on dist-test-slave-gjrz"
server_key: "b27356ea3fa0dfc6b414a726ffc65bde"
server_key_iv: "2d3f7fd1ca36bd33445089b86243dc10"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.098049 13989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:36.098698 14012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.098866 13989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:36.098960 13989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "21b2d70740154f889b47154f7522d23c"
format_stamp: "Formatted at 2026-08-06 22:40:36 on dist-test-slave-gjrz"
server_key: "b27356ea3fa0dfc6b414a726ffc65bde"
server_key_iv: "2d3f7fd1ca36bd33445089b86243dc10"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.099028 13989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:36.117630 13989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:36.119473 13989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:36.119582 13989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:36.119762 13989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:36.120057 13989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:36.120098 13989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.120128 13989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:36.120153 13989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.125756 13989 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35735
I20260806 22:40:36.125810 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35735 every 8 connection(s)
I20260806 22:40:36.126106 13989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:36.134546 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.126803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:52509 (local address 127.0.186.254:36759)
0806 22:40:36.126849 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.126851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.127031 (+ 180us) server_negotiation.cc:407] Connection header received
0806 22:40:36.127955 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.127956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.127966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.128040 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.128589 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.129094 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.130038 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.130233 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.130288 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.132264 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:36.132265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:36.132266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:36.132272 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:36.133520 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:36.133851 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:36.133852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:36.133852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:36.133902 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:36.134080 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:36.134081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:36.134081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:36.134177 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:36.134223 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:36.134415 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:40:36.134477 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:40:36.134711 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.126765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:52509)
0806 22:40:36.126910 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.126950 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.127875 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.128104 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.128108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.128346 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.128526 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.128529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.129160 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.129160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.129930 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.129932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.130295 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.130295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.130396 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.130798 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.130806 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.132166 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:36.133590 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:36.133595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:36.133597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:36.133791 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:36.133946 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:36.133946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:36.133947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:36.134039 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:36.134218 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:36.134221 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:36.134324 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:40:36.134399 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:40:36.134511 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260806 22:40:36.135192 14183 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36759
I20260806 22:40:36.135315 14183 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:36.135499 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 13989
I20260806 22:40:36.135536 14183 heartbeater.cc:511] Master 127.0.186.254:36759 requested a full tablet report, sending...
I20260806 22:40:36.135596 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:36.135762 747 external_mini_cluster.cc:1428] Setting key 98597cc0158af5ec9e3e8d0cd5ec71f4
I20260806 22:40:36.135995 13522 ts_manager.cc:194] Registered new tserver with Master: 21b2d70740154f889b47154f7522d23c (127.0.186.195:35735)
I20260806 22:40:36.136473 13522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:52509
I20260806 22:40:36.136596 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
Entry for principal oryx/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab.
WARNING: no policy specified for oryx/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
Entry for principal oryx/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab.
WARNING: no policy specified for oryx/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
Entry for principal oryx/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab.
I20260806 22:40:36.203751 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 13606
I20260806 22:40:36.214825 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 13800
I20260806 22:40:36.234077 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 13989
I20260806 22:40:36.248759 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 13469
I20260806 22:40:36.258658 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36759
--webserver_interface=127.0.186.254
--webserver_port=41775
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:36759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:36.341223 14198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:36.341427 14198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:36.341449 14198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:36.342615 14198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:36.342658 14198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:36.342671 14198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:36.342682 14198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:36.342694 14198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:36.344235 14198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:36759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=41775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:36.344466 14198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:36.344676 14198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:36.349633 14209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.349651 14204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.349633 14207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.349915 14198 server_base.cc:1034] running on GCE node
I20260806 22:40:36.350162 14198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:36.350477 14198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:36.351640 14198 hybrid_clock.cc:648] HybridClock initialized: now 1786056036351624 us; error 37 us; skew 500 ppm
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:36.354168 14198 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:36.354619 14198 webserver.cc:460] Webserver started at http://127.0.186.254:41775/ using document root <none> and password file <none>
I20260806 22:40:36.354815 14198 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:36.354871 14198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:36.356261 14198 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:36.356907 14217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.357060 14198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:36.357147 14198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "7866d9ab19df49c59ab3e8e53989ca81"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "a02bffac27b24e9c9c28f5de716a7463"
server_key_iv: "7c4381df32ac186c91916e659d6f0731"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.357504 14198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:36.375094 14198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:36.377221 14198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:36.377347 14198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:36.381428 14198 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:36759
I20260806 22:40:36.381471 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:36759 every 8 connection(s)
I20260806 22:40:36.381966 14198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:36.383224 14294 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:36.383913 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 14198
I20260806 22:40:36.383939 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Bootstrap starting.
I20260806 22:40:36.384173 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:40231
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:36759
--webserver_port=40181
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:36.392093 14294 log.cc:824] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:36.393684 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:36.393929 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Bootstrap complete.
I20260806 22:40:36.396060 14294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:36.396279 14294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7866d9ab19df49c59ab3e8e53989ca81, State: Initialized, Role: FOLLOWER
I20260806 22:40:36.396430 14294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:36.396505 14294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:36.396556 14294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:36.396601 14294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:40:36.397548 14294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:36.397646 14294 leader_election.cc:302] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7866d9ab19df49c59ab3e8e53989ca81; no voters:
I20260806 22:40:36.397806 14294 leader_election.cc:288] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:40:36.397853 14302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:40:36.398025 14302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [term 2 LEADER]: Becoming Leader. State: Replica: 7866d9ab19df49c59ab3e8e53989ca81, State: Running, Role: LEADER
I20260806 22:40:36.398080 14294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:36.398276 14302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } }
I20260806 22:40:36.398635 14305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7866d9ab19df49c59ab3e8e53989ca81. Latest consensus state: current_term: 2 leader_uuid: "7866d9ab19df49c59ab3e8e53989ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } } }
I20260806 22:40:36.398720 14305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:36.398897 14304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7866d9ab19df49c59ab3e8e53989ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7866d9ab19df49c59ab3e8e53989ca81" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36759 } } }
I20260806 22:40:36.398960 14304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:36.399425 14312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:36.399978 14312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:36.400101 14312 catalog_manager.cc:1259] Loaded cluster ID: 75c3fa0d23424c4dbadfae59fdc1b144
I20260806 22:40:36.400133 14312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:36.401348 14312 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:36.401530 14312 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7866d9ab19df49c59ab3e8e53989ca81: Loaded TSK: 0
I20260806 22:40:36.401741 14312 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:40:36.468041 14296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:36.468252 14296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:36.468276 14296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:36.469527 14296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:36.469619 14296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:36.469668 14296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:36.471175 14296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.193.keytab
--principal=oryx/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:40231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=40181
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:36.471426 14296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:36.471653 14296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:36.477824 14334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.477821 14336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.478065 14296 server_base.cc:1034] running on GCE node
W20260806 22:40:36.478148 14339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.478356 14296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:36.478643 14296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:36.479823 14296 hybrid_clock.cc:648] HybridClock initialized: now 1786056036479803 us; error 33 us; skew 500 ppm
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:36.482930 14296 init.cc:377] Logged in from keytab as oryx/127.0.186.193@KRBTEST.COM (short username oryx)
I20260806 22:40:36.483505 14296 webserver.cc:460] Webserver started at http://127.0.186.193:40181/ using document root <none> and password file <none>
I20260806 22:40:36.483758 14296 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:36.483841 14296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:36.485558 14296 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:36.486290 14353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.486486 14296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:36.486580 14296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "8950c889657a4bbbb401a265367c2cfd"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "bd77a2debb519e12f0c41f1e7563269f"
server_key_iv: "4b473f19bcd22a99aaa3cf29c83fb28c"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.486941 14296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:36.506618 14296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:36.508844 14296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:36.508962 14296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:36.509202 14296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:36.509517 14296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:36.509599 14296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.509639 14296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:36.509655 14296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.515434 14296 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:40231
I20260806 22:40:36.515491 14520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:40231 every 8 connection(s)
I20260806 22:40:36.515851 14296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:36.519802 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 14296
I20260806 22:40:36.520079 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:43619
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:36759
--webserver_port=34131
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.525252 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.516642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:43799)
0806 22:40:36.516798 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.516842 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.517307 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.517760 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.517763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.518348 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.518545 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.518551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.520158 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.520160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.521247 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.521251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.522136 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.522137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.522272 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.522800 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.522810 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.525056 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
W20260806 22:40:36.525535 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.516696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:43799 (local address 127.0.186.254:36759)
0806 22:40:36.516902 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.516910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.516946 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:40:36.517516 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.517524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.517581 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.517734 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.518687 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.520031 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.521335 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.521943 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.522042 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.525198 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:43799: BlockingRecv error: recv got EOF from 127.0.186.193:43799 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
W20260806 22:40:36.525542 14521 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36759 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.186.254:36759: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.529891 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.525764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:52813)
0806 22:40:36.525846 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.525857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.525908 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.526030 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.526031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.526152 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.526195 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.526197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.526776 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.526777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.527302 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.527304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.527613 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.527614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.527675 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.528024 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.528029 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.529822 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 22:40:36.529971 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.525806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:52813 (local address 127.0.186.254:36759)
0806 22:40:36.525848 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.525849 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.525905 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:40:36.525965 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.525965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.525976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.526029 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.526257 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.526711 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.527364 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.527555 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.527609 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.529922 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:52813: BlockingRecv error: recv got EOF from 127.0.186.193:52813 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.534479 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.530215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:41893)
0806 22:40:36.530270 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.530282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.530332 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.530464 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.530465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.530599 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.530635 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.530637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.531215 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.531216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.531992 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.531995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.532355 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.532356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.532410 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.532750 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.532756 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.534417 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:40:36.534530 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.530266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:41893 (local address 127.0.186.254:36759)
0806 22:40:36.530311 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.530313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.530322 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:36.530393 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.530394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.530401 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.530456 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.530701 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.531142 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.532092 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.532278 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.532340 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.534489 (+ 2149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:41893: BlockingRecv error: recv got EOF from 127.0.186.193:41893 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:40:36.534680 14521 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:40:36.605821 14526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:36.606043 14526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:36.606074 14526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:36.607234 14526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:36.607290 14526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:36.607326 14526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:36.608767 14526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.194.keytab
--principal=oryx/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:43619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=34131
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:36.609005 14526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:36.609220 14526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:36.614197 14534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.614197 14532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.614217 14536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.614444 14526 server_base.cc:1034] running on GCE node
I20260806 22:40:36.614646 14526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:36.614900 14526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:36.616079 14526 hybrid_clock.cc:648] HybridClock initialized: now 1786056036616042 us; error 29 us; skew 500 ppm
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:36.618727 14526 init.cc:377] Logged in from keytab as oryx/127.0.186.194@KRBTEST.COM (short username oryx)
I20260806 22:40:36.619163 14526 webserver.cc:460] Webserver started at http://127.0.186.194:34131/ using document root <none> and password file <none>
I20260806 22:40:36.619421 14526 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:36.619488 14526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:36.620751 14526 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:36.621371 14547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.621531 14526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:36.621611 14526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "51d338dcc46048f99a57f8862e1d93eb"
format_stamp: "Formatted at 2026-08-06 22:40:35 on dist-test-slave-gjrz"
server_key: "539a1f0b4c898e4673ca8bf822e8b297"
server_key_iv: "0f9abbf186eadaa8d2667b045d310189"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.622004 14526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:36.641373 14526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:36.643512 14526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:36.643647 14526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:36.643872 14526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:36.644176 14526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:36.644207 14526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.644248 14526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:36.644296 14526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.650209 14526 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43619
I20260806 22:40:36.650249 14726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43619 every 8 connection(s)
I20260806 22:40:36.650571 14526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.657159 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 14526
I20260806 22:40:36.657421 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:35735
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:36759
--webserver_port=34035
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:36.657469 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.651224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:34907 (local address 127.0.186.254:36759)
0806 22:40:36.651289 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.651291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.651502 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:40:36.652013 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.652016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.652034 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.652101 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.653030 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.653524 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.654484 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.654680 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.654736 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.657412 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:34907: BlockingRecv error: recv got EOF from 127.0.186.194:34907 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:40:36.657510 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.651204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:34907)
0806 22:40:36.651386 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.651423 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.651914 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.652186 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.652190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.652740 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.652932 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.652936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.653610 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.653611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.654397 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.654399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.654750 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.654751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.654886 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.655341 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.655349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.657321 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
W20260806 22:40:36.657763 14727 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36759 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.186.254:36759: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.662071 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.657949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:33889)
0806 22:40:36.657990 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.657998 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.658030 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.658164 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.658164 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.658284 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.658326 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.658327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.658905 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.658906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.659455 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.659457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.659823 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.659823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.659891 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.660251 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.660256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.662012 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:40:36.662165 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.658014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:33889 (local address 127.0.186.254:36759)
0806 22:40:36.658066 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.658068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.658075 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:36.658098 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.658099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.658107 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.658154 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.658388 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.658840 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.659530 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.659757 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.659804 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.662113 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:33889: BlockingRecv error: recv got EOF from 127.0.186.194:33889 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.666030 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.662286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:44639)
0806 22:40:36.662406 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.662414 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.662460 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.662571 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.662571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.662685 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.662711 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.662712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.663298 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.663299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.663775 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.663776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.664095 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.664095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.664146 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.664475 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.664480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.665970 (+ 1490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260806 22:40:36.666096 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.662354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:44639 (local address 127.0.186.254:36759)
0806 22:40:36.662384 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.662385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.662460 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:40:36.662513 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.662514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.662522 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.662566 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.662771 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.663214 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.663844 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.664028 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.664083 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.666039 (+ 1956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:44639: BlockingRecv error: recv got EOF from 127.0.186.194:44639 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 22:40:36.666157 14727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:40:36.738222 14731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:36.738404 14731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:36.738426 14731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:36.739616 14731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:36.739656 14731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:36.739681 14731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:36.741089 14731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/krb5kdc/oryx_127.0.186.195.keytab
--principal=oryx/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:35735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=34035
--tserver_master_addrs=127.0.186.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:36.741292 14731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:36.741488 14731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:36.745932 14741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.745939 14739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:36.746073 14738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:36.746378 14731 server_base.cc:1034] running on GCE node
I20260806 22:40:36.746567 14731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:36.746776 14731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:36.747903 14731 hybrid_clock.cc:648] HybridClock initialized: now 1786056036747889 us; error 23 us; skew 500 ppm
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056036, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:36.750196 14731 init.cc:377] Logged in from keytab as oryx/127.0.186.195@KRBTEST.COM (short username oryx)
I20260806 22:40:36.750602 14731 webserver.cc:460] Webserver started at http://127.0.186.195:34035/ using document root <none> and password file <none>
I20260806 22:40:36.750797 14731 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:36.750860 14731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:36.752140 14731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:36.752655 14755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.752801 14731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:36.752880 14731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "21b2d70740154f889b47154f7522d23c"
format_stamp: "Formatted at 2026-08-06 22:40:36 on dist-test-slave-gjrz"
server_key: "b27356ea3fa0dfc6b414a726ffc65bde"
server_key_iv: "2d3f7fd1ca36bd33445089b86243dc10"
server_key_version: "encryptionkey@0"
I20260806 22:40:36.753193 14731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:36.767902 14731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:36.769610 14731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:36.769740 14731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:36.769932 14731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:36.770269 14731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:36.770306 14731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.770326 14731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:36.770340 14731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:36.775779 14731 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35735
I20260806 22:40:36.775825 14939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35735 every 8 connection(s)
I20260806 22:40:36.776154 14731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.782646 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.776786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:47737)
0806 22:40:36.776932 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.776964 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.777434 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.777626 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.777629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.778165 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.778343 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.778348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.778958 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.778959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.779761 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.779763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.780124 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.780125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.780217 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.780583 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.780591 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.782516 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
W20260806 22:40:36.782665 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.776778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:47737 (local address 127.0.186.254:36759)
0806 22:40:36.776829 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.776831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.777045 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:40:36.777509 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.777510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.777521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.777570 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.778418 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.778896 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.779833 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.780047 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.780100 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.782619 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:47737: BlockingRecv error: recv got EOF from 127.0.186.195:47737 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:40:36.782881 14941 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36759 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.186.254:36759: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
I20260806 22:40:36.783028 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 14731
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.787294 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.783098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:45817)
0806 22:40:36.783136 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.783146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.783184 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.783543 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.783544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.783684 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.783737 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.783739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.784327 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.784327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.784833 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.784835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.785149 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.785149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.785214 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.785549 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.785553 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.787211 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 22:40:36.787374 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.783291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:45817 (local address 127.0.186.254:36759)
0806 22:40:36.783328 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.783329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.783336 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:36.783354 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.783355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.783362 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.783416 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.783810 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.784255 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.784911 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.785083 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.785135 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.787318 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:45817: BlockingRecv error: recv got EOF from 127.0.186.195:45817 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:36.790403 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.783439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.0.186.254:36759)
0806 22:40:36.783546 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.783548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.783622 (+ 74us) server_negotiation.cc:407] Connection header received
0806 22:40:36.783699 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.783699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.783708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.783755 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.784332 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.784767 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.785635 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.785815 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.785862 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.787144 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:36.787149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:36.787150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:36.787161 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:36.788996 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:36.789342 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:36.789345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:36.789345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:36.789441 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:36.789620 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:36.789620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:36.789621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:36.789904 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:36.790045 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:36.790203 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:40:36.790311 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:36 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:36.791728 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:36.787532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:57481)
0806 22:40:36.787570 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:36.787588 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:36.787620 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:36.787740 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:36.787741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:36.787860 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:36.787890 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.787891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.788463 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.788464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.789196 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:36.789198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.789600 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.789601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:36.789653 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.789976 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:36.789981 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:36.791687 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:40:36.791800 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:36.787575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:57481 (local address 127.0.186.254:36759)
0806 22:40:36.787618 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:36.787620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:36.787628 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:36.787671 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:36.787671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:36.787678 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:36.787721 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:36.787960 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.788376 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.789266 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:36.789520 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:36.789581 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:36.791751 (+ 2170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:57481: BlockingRecv error: recv got EOF from 127.0.186.195:57481 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:40:36.791910 14941 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:37.539342 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:37.535041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:44497)
0806 22:40:37.535153 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:37.535167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:37.535234 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:37.535396 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:37.535397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:37.535555 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:37.535607 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.535608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.536208 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.536209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.536737 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.536738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.537024 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.537025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.537079 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.537543 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:37.537549 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:37.539225 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
W20260806 22:40:37.539398 14949 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:37.535112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:44497 (local address 127.0.186.254:36759)
0806 22:40:37.535254 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:37.535256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:37.535286 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:40:37.535314 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:37.535315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:37.535326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:37.535384 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:37.535660 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.536152 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.536789 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.536963 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.537007 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.539337 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:44497: BlockingRecv error: recv got EOF from 127.0.186.193:44497 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:37.671146 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:37.666620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:39657)
0806 22:40:37.666800 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:37.666814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:37.666886 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:37.667057 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:37.667058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:37.667311 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:37.667376 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.667379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.668036 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.668038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.668591 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.668593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.668965 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.668966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.669012 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.669419 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:37.669423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:37.671074 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
W20260806 22:40:37.671289 14949 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:37.666713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:39657 (local address 127.0.186.254:36759)
0806 22:40:37.666791 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:37.666793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:37.666879 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:40:37.666946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:37.666947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:37.666960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:37.667038 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:37.667443 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.667960 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.668690 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.668891 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.668954 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.671203 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:39657: BlockingRecv error: recv got EOF from 127.0.186.194:39657 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:37 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:37.797466 14952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:37.792324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:47031)
0806 22:40:37.792460 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:37.792475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:37.792548 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:37.792688 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:37.792689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:37.792868 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:37.792919 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.792922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.793626 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.793627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.794175 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:37.794177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.794518 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.794519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:37.794576 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.795051 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:37.795056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:37.797395 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260806 22:40:37.797574 14949 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:37.792418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:47031 (local address 127.0.186.254:36759)
0806 22:40:37.792490 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:37.792492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:37.792529 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:40:37.792611 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:37.792612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:37.792625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:37.792672 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:37.793002 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.793512 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.794260 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:37.794453 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:37.794510 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:37.797496 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:47031: BlockingRecv error: recv got EOF from 127.0.186.195:47031 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:38.544129 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.539785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:50163)
0806 22:40:38.539907 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.539920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:38.540001 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:38.540157 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:38.540157 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:38.540321 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:38.540372 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.540373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.540973 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.540974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.541533 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.541535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.541820 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.541821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.541855 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.542336 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:38.542340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:38.544070 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
W20260806 22:40:38.544260 14956 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:38.539901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:50163 (local address 127.0.186.254:36759)
0806 22:40:38.540029 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:38.540032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:38.540039 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:38.540060 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:38.540061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:38.540070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:38.540142 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:38.540424 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.540910 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.541592 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.541775 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.541817 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.544190 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:50163: BlockingRecv error: recv got EOF from 127.0.186.193:50163 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:38.676268 14958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.671657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:36723)
0806 22:40:38.671805 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.671817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:38.671881 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:38.672079 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:38.672080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:38.672247 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:38.672323 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.672325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.672999 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.673000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.673623 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.673627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.673980 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.673981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.674031 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.674450 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:38.674455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:38.676208 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
W20260806 22:40:38.676390 14956 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:38.671761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:36723 (local address 127.0.186.254:36759)
0806 22:40:38.671826 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:38.671828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:38.671873 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:40:38.671927 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:38.671928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:38.671938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:38.671983 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:38.672391 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.672893 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.673701 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.673897 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.673949 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.676327 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:36723: BlockingRecv error: recv got EOF from 127.0.186.194:36723 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:38 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:38.802429 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.797925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:40695)
0806 22:40:38.798047 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.798058 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:38.798115 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:38.798283 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:38.798284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:38.798472 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:38.798523 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.798525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.799170 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.799172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.799786 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.799789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.800141 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.800143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.800197 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.800640 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:38.800645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:38.802358 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
W20260806 22:40:38.802537 14956 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:38.798074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:40695 (local address 127.0.186.254:36759)
0806 22:40:38.798127 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:38.798128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:38.798138 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:38.798164 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:38.798165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:38.798176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:38.798254 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:38.798582 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.799094 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.799858 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.800050 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.800112 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.802468 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:40695: BlockingRecv error: recv got EOF from 127.0.186.195:40695 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:39.549168 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.544627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:42171)
0806 22:40:39.544786 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.544800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.544859 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.544991 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.544991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.545154 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.545204 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.545205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.545840 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.545842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.546393 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.546396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.546858 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.546860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.546927 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.547387 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.547393 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.549089 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
W20260806 22:40:39.549256 14964 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:39.544707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:42171 (local address 127.0.186.254:36759)
0806 22:40:39.544838 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.544840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.544848 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:39.544911 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.544912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.544923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.544979 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.545256 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.545763 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.546482 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.546783 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.546844 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.549194 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:42171: BlockingRecv error: recv got EOF from 127.0.186.193:42171 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:39.681370 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.676739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:38099)
0806 22:40:39.676891 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.676903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.676973 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.677115 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.677116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.677274 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.677329 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.677331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.678043 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.678045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.678606 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.678609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.678967 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.678968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.679013 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.679466 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.679471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.681309 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
W20260806 22:40:39.681510 14964 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:39.676795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:38099 (local address 127.0.186.254:36759)
0806 22:40:39.676862 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.676865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.677009 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:40:39.677040 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.677041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.677054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.677101 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.677424 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.677952 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.678702 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.678901 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.678957 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.681439 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:38099: BlockingRecv error: recv got EOF from 127.0.186.194:38099 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:39 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:39.807663 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.802960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:44241)
0806 22:40:39.803160 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.803173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.803240 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.803402 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.803403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.803575 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.803623 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.803625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.804305 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.804307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.804839 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.804841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.805218 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.805219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.805273 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.805706 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.805711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.807581 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
W20260806 22:40:39.807713 14964 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:39.803026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:44241 (local address 127.0.186.254:36759)
0806 22:40:39.803120 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.803122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.803255 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:40:39.803313 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.803314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.803330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.803383 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.803706 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.804234 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.804935 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.805133 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.805189 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.807659 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:44241: BlockingRecv error: recv got EOF from 127.0.186.195:44241 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:40.554111 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.549669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:47657)
0806 22:40:40.549799 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.549810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.549862 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.550007 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.550008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.550165 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.550213 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.550215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.550838 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.550839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.551442 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.551445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.551851 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.551852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.551935 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.552356 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.552361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.554050 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260806 22:40:40.554222 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.549716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:47657 (local address 127.0.186.254:36759)
0806 22:40:40.549841 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.549843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.549854 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:40.549924 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.549925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.549936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.549994 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.550272 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.550760 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.551525 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.551763 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.551823 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.554144 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:47657: BlockingRecv error: recv got EOF from 127.0.186.193:47657 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:40.690923 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.681796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:37689)
0806 22:40:40.681931 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.681942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.682006 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.682175 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.682176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.682335 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.682384 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.682386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.682969 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.682970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.683563 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.683566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.683881 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.683881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.683918 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.689097 (+ 5179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.689102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.690848 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":56,"threads_started":1}
W20260806 22:40:40.690999 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.681877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:37689 (local address 127.0.186.254:36759)
0806 22:40:40.681945 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.681947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.681990 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:40:40.682058 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.682059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.682069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.682121 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.682444 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.682917 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.683640 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.683830 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.683876 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.690946 (+ 7070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:37689: BlockingRecv error: recv got EOF from 127.0.186.194:37689 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:40 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:40.812520 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.808159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:52859)
0806 22:40:40.808303 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.808314 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.808368 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.808503 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.808504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.808698 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.808759 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.808762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.809403 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.809404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.809930 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.809932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.810222 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.810223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.810261 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.810693 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.810698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.812460 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
W20260806 22:40:40.812610 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.808219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:52859 (local address 127.0.186.254:36759)
0806 22:40:40.808263 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.808266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.808356 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:40:40.808418 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.808418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.808433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.808491 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.808830 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.809331 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.809990 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.810175 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.810218 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.812546 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:52859: BlockingRecv error: recv got EOF from 127.0.186.195:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:41.559245 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.554559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:57949)
0806 22:40:41.554680 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:41.554693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:41.554755 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:41.554914 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:41.554915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:41.555081 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:41.555134 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.555136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.555895 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.555896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.556542 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.556545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.556892 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.556893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.556942 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.557405 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:41.557410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:41.559164 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260806 22:40:41.559353 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.554646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:57949 (local address 127.0.186.254:36759)
0806 22:40:41.554801 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.554803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.554811 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:41.554833 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.554834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.554844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.554903 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.555203 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.555802 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.556607 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.556827 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.556881 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.559289 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:57949: BlockingRecv error: recv got EOF from 127.0.186.193:57949 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:41.695932 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.691415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:45223)
0806 22:40:41.691538 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:41.691550 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:41.691609 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:41.691723 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:41.691724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:41.691882 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:41.691931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.691934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.692589 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.692590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.693136 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.693138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.693521 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.693522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.693578 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.694084 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:41.694090 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:41.695869 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
W20260806 22:40:41.696043 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.691512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45223 (local address 127.0.186.254:36759)
0806 22:40:41.691565 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.691567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.691603 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:40:41.691653 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.691653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.691664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.691713 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.692003 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.692501 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.693220 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.693431 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.693483 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.695981 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:45223: BlockingRecv error: recv got EOF from 127.0.186.194:45223 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:41 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:41.817480 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.812985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:44907)
0806 22:40:41.813138 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:41.813148 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:41.813204 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:41.813327 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:41.813328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:41.813502 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:41.813559 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.813561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.814194 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.814195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.814726 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:41.814727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.815043 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.815043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:41.815081 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.815511 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:41.815515 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:41.817395 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
W20260806 22:40:41.817553 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.813074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:44907 (local address 127.0.186.254:36759)
0806 22:40:41.813130 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.813132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.813199 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:40:41.813260 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.813260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.813272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.813313 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.813622 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.814131 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.814794 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.814977 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.815042 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.817491 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:44907: BlockingRecv error: recv got EOF from 127.0.186.195:44907 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:42.564606 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.559764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:46373)
0806 22:40:42.559906 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.559920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.559993 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.560124 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.560125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.560281 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:42.560333 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.560334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.561059 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.561061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.561762 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.561765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.562111 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.562112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.562170 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.562625 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:42.562630 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:42.564545 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
W20260806 22:40:42.564735 14981 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:42.559856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:46373 (local address 127.0.186.254:36759)
0806 22:40:42.560001 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.560003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.560014 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:42.560038 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.560039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.560055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.560113 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:42.560426 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.560924 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.561833 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.562048 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.562102 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.564654 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:46373: BlockingRecv error: recv got EOF from 127.0.186.193:46373 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:42.700704 14982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.696387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:50501)
0806 22:40:42.696505 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.696517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.696580 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.696695 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.696696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.696853 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:42.696902 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.696904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.697510 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.697511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.698089 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.698093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.698392 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.698393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.698451 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.698858 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:42.698862 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:42.700646 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
W20260806 22:40:42.700805 14981 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:42.696454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50501 (local address 127.0.186.254:36759)
0806 22:40:42.696532 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.696535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.696563 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:40:42.696624 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.696625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.696636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.696682 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:42.696953 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.697440 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.698152 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.698339 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.698389 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.700740 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:50501: BlockingRecv error: recv got EOF from 127.0.186.194:50501 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:42 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:42.822521 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.817941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:45285)
0806 22:40:42.818102 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.818114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.818177 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.818303 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.818304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.818459 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:42.818510 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.818511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.819096 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.819096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.819630 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.819631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.820079 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.820080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.820129 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.820594 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:42.820598 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:42.822463 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
W20260806 22:40:42.822621 14981 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:42.818039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:45285 (local address 127.0.186.254:36759)
0806 22:40:42.818104 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.818106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.818167 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:40:42.818219 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.818220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.818233 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.818288 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:42.818558 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.819043 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.819705 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.819958 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.820062 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.822560 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:45285: BlockingRecv error: recv got EOF from 127.0.186.195:45285 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:43.569628 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.565077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:56415)
0806 22:40:43.565204 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.565215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.565268 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.565407 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.565407 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.565585 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:43.565638 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.565640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.566356 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.566358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.566936 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.566939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.567300 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.567302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.567366 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.567764 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:43.567768 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:43.569567 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
W20260806 22:40:43.569729 14985 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:43.565114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:56415 (local address 127.0.186.254:36759)
0806 22:40:43.565245 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.565247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.565258 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:43.565320 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.565320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.565332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.565395 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.565688 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.566270 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.567003 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.567204 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.567254 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.569661 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:56415: BlockingRecv error: recv got EOF from 127.0.186.193:56415 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:43.705826 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.701155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:35795)
0806 22:40:43.701291 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.701306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.701381 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.701530 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.701531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.701702 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:43.701751 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.701753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.702406 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.702407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.702952 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.702954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.703320 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.703321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.703379 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.703789 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:43.703793 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:43.705747 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260806 22:40:43.705912 14985 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:43.701247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:35795 (local address 127.0.186.254:36759)
0806 22:40:43.701330 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.701332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.701359 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:40:43.701453 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.701454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.701467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.701515 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.701826 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.702330 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.703038 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.703230 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.703310 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.705858 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:35795: BlockingRecv error: recv got EOF from 127.0.186.194:35795 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:43 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:43.827493 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.822997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:51027)
0806 22:40:43.823187 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.823198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.823257 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.823408 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.823409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.823565 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:43.823621 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.823622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.824292 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.824294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.824825 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.824827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.825139 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.825140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.825198 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.825641 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:43.825646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:43.827416 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
W20260806 22:40:43.827564 14985 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:43.823070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:51027 (local address 127.0.186.254:36759)
0806 22:40:43.823148 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.823150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.823253 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:40:43.823336 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.823336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.823348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.823396 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.823672 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.824234 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.824886 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.825070 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.825117 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.827510 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:51027: BlockingRecv error: recv got EOF from 127.0.186.195:51027 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:44.574281 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:44.570069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:42481)
0806 22:40:44.570206 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:44.570218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:44.570265 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:44.570420 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:44.570421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:44.570583 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:44.570634 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.570636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.571233 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.571233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.571784 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.571785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.572096 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.572097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.572142 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.572542 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:44.572547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:44.574212 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
W20260806 22:40:44.574378 14994 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:44.570182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:42481 (local address 127.0.186.254:36759)
0806 22:40:44.570307 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:44.570309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:44.570317 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:44.570340 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:44.570340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:44.570350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:44.570407 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:44.570684 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.571178 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.571840 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.572022 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.572065 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.574312 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:42481: BlockingRecv error: recv got EOF from 127.0.186.193:42481 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:44.711144 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:44.706329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:43399)
0806 22:40:44.706469 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:44.706481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:44.706540 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:44.706662 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:44.706662 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:44.706834 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:44.706892 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.706894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.707710 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.707711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.708559 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.708563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.708899 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.708901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.708954 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.709355 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:44.709360 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:44.711075 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
W20260806 22:40:44.711230 14994 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:44.706414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:43399 (local address 127.0.186.254:36759)
0806 22:40:44.706472 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:44.706474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:44.706534 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:40:44.706586 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:44.706587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:44.706598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:44.706647 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:44.706951 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.707633 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.708628 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.708823 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.708875 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.711169 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:43399: BlockingRecv error: recv got EOF from 127.0.186.194:43399 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:44 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:44.832182 14996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:44.828026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:33519)
0806 22:40:44.828189 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:44.828201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:44.828262 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:44.828386 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:44.828387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:44.828548 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:44.828600 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.828603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.829224 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.829225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.829753 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:44.829754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.830047 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.830048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:44.830090 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.830505 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:44.830509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:44.832120 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
W20260806 22:40:44.832356 14994 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:44.828095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:33519 (local address 127.0.186.254:36759)
0806 22:40:44.828165 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:44.828168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:44.828257 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:40:44.828317 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:44.828318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:44.828331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:44.828368 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:44.828660 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.829160 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.829812 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:44.829997 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:44.830046 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:44.832276 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:33519: BlockingRecv error: recv got EOF from 127.0.186.195:33519 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:45.579116 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:45.574708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:47321)
0806 22:40:45.574848 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:45.574859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:45.574919 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:45.575034 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:45.575035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:45.575220 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:45.575287 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.575289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.575955 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.575956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.576489 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.576491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.576825 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.576826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.576879 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.577315 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:45.577319 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:45.579055 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
W20260806 22:40:45.579214 14998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:45.574803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:47321 (local address 127.0.186.254:36759)
0806 22:40:45.574919 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:45.574921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:45.574930 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:45.574957 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:45.574957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:45.574968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:45.575021 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:45.575346 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.575849 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.576552 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.576746 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.576791 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.579145 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:47321: BlockingRecv error: recv got EOF from 127.0.186.193:47321 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:45.728422 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:45.711604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:56815)
0806 22:40:45.711732 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:45.711742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:45.711816 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:45.711939 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:45.711940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:45.712145 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:45.712195 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.712197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.712868 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.712869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.713394 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.713396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.713742 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.713744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.713802 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.726440 (+ 12638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:45.726446 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:45.728346 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260806 22:40:45.728528 14998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:45.711649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:56815 (local address 127.0.186.254:36759)
0806 22:40:45.711693 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:45.711695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:45.711816 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:40:45.711874 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:45.711875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:45.711888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:45.711935 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:45.712277 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.712787 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.713475 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.713677 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.713736 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.728471 (+ 14735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:56815: BlockingRecv error: recv got EOF from 127.0.186.194:56815 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:45 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:45.837198 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:45.832606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:35171)
0806 22:40:45.832729 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:45.832740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:45.832799 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:45.832919 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:45.832920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:45.833091 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:45.833137 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.833139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.833805 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.833806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.834342 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:45.834344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.834720 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.834721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:45.834786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.835245 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:45.835250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:45.837128 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
W20260806 22:40:45.837278 14998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:45.832667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:35171 (local address 127.0.186.254:36759)
0806 22:40:45.832741 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:45.832743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:45.832794 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:40:45.832852 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:45.832853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:45.832866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:45.832905 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:45.833187 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.833704 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.834431 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:45.834630 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:45.834686 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:45.837220 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:35171: BlockingRecv error: recv got EOF from 127.0.186.195:35171 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:46.584103 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:46.579622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:52751)
0806 22:40:46.579770 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:46.579781 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:46.579831 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:46.579968 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:46.579969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:46.580157 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:46.580210 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.580212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.580839 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.580840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.581388 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.581389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.581726 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.581727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.581773 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.582294 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:46.582299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:46.584034 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
W20260806 22:40:46.584216 15003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:46.579689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:52751 (local address 127.0.186.254:36759)
0806 22:40:46.579810 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:46.579812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:46.579821 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:46.579893 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:46.579894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:46.579905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:46.579951 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:46.580265 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.580764 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.581458 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.581662 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.581711 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.584146 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:52751: BlockingRecv error: recv got EOF from 127.0.186.193:52751 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:46.733446 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:46.728896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:60563)
0806 22:40:46.729038 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:46.729051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:46.729129 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:46.729264 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:46.729265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:46.729435 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:46.729481 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.729483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.730144 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.730145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.730656 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.730657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.730980 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.730981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.731033 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.731643 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:46.731649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:46.733387 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
W20260806 22:40:46.733592 15003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:46.728989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:60563 (local address 127.0.186.254:36759)
0806 22:40:46.729048 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:46.729051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:46.729109 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:40:46.729189 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:46.729190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:46.729205 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:46.729251 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:46.729551 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.730054 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.730728 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.730918 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.730976 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.733529 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:60563: BlockingRecv error: recv got EOF from 127.0.186.194:60563 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:46 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:46.842074 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:46.837657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:37909)
0806 22:40:46.837826 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:46.837842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:46.837919 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:46.838057 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:46.838058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:46.838230 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:46.838278 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.838280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.838944 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.838947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.839533 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:46.839536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.839870 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.839871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:46.839928 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.840337 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:46.840342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:46.842005 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
W20260806 22:40:46.842228 15003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:46.837806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:37909 (local address 127.0.186.254:36759)
0806 22:40:46.837862 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:46.837864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:46.837897 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:40:46.837978 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:46.837979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:46.837993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:46.838038 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:46.838351 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.838868 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.839616 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:46.839805 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:46.839862 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:46.842164 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:37909: BlockingRecv error: recv got EOF from 127.0.186.195:37909 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:47.588786 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:47.584548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:57601)
0806 22:40:47.584673 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:47.584683 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:47.584733 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:47.584857 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:47.584858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:47.585027 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:47.585076 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.585078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.585711 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.585711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.586249 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.586250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.586594 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.586595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.586632 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.587023 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:47.587028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:47.588711 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
W20260806 22:40:47.588876 15007 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:47.584616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:57601 (local address 127.0.186.254:36759)
0806 22:40:47.584730 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:47.584732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:47.584740 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:47.584777 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:47.584777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:47.584787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:47.584846 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:47.585137 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.585649 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.586333 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.586529 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.586586 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.588809 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:57601: BlockingRecv error: recv got EOF from 127.0.186.193:57601 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:47.738313 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:47.733886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:59163)
0806 22:40:47.734035 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:47.734048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:47.734119 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:47.734243 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:47.734243 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:47.734431 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:47.734478 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.734480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.735129 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.735130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.735701 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.735703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.736036 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.736037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.736089 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.736519 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:47.736523 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:47.738245 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
W20260806 22:40:47.738428 15007 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:47.733989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:59163 (local address 127.0.186.254:36759)
0806 22:40:47.734048 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:47.734050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:47.734109 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:40:47.734174 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:47.734176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:47.734187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:47.734232 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:47.734554 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.735053 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.735782 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.735975 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.736032 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.738344 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:59163: BlockingRecv error: recv got EOF from 127.0.186.194:59163 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:47 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:47.847252 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:47.842546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:47611)
0806 22:40:47.842739 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:47.842754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:47.842829 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:47.842957 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:47.842958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:47.843152 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:47.843199 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.843201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.843883 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.843884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.844459 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:47.844462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.844798 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.844799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:47.844853 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.845301 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:47.845306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:47.847158 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":102,"threads_started":1}
W20260806 22:40:47.847327 15007 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:47.842653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:47611 (local address 127.0.186.254:36759)
0806 22:40:47.842744 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:47.842747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:47.842821 (+ 74us) server_negotiation.cc:407] Connection header received
0806 22:40:47.842884 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:47.842885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:47.842900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:47.842950 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:47.843296 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.843805 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.844543 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:47.844738 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:47.844793 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:47.847259 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:47611: BlockingRecv error: recv got EOF from 127.0.186.195:47611 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:48.594125 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:48.589312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:50607)
0806 22:40:48.589440 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:48.589452 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:48.589505 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:48.589661 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:48.589662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:48.589843 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:48.589893 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.589895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.590590 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.590591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.591222 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.591225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.591672 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.591673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.591733 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.592220 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:48.592225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:48.594057 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
W20260806 22:40:48.594220 15015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:48.589402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:50607 (local address 127.0.186.254:36759)
0806 22:40:48.589533 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:48.589535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:48.589544 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:48.589570 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:48.589571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:48.589582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:48.589653 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:48.590005 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.590518 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.591356 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.591559 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.591636 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.594153 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:50607: BlockingRecv error: recv got EOF from 127.0.186.193:50607 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:48.743095 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:48.738794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:35429)
0806 22:40:48.738934 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:48.738944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:48.739000 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:48.739116 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:48.739116 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:48.739317 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:48.739366 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.739368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.740006 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.740007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.740522 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.740523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.740877 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.740879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.740940 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.741348 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:48.741352 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:48.743032 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
W20260806 22:40:48.743191 15015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:48.738865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:35429 (local address 127.0.186.254:36759)
0806 22:40:48.738930 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:48.738932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:48.738994 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:40:48.739045 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:48.739046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:48.739057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:48.739110 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:48.739420 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.739944 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.740598 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.740786 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.740835 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.743134 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:35429: BlockingRecv error: recv got EOF from 127.0.186.194:35429 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:48 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:48.852402 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:48.847760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:34835)
0806 22:40:48.847902 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:48.847914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:48.847984 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:48.848140 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:48.848141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:48.848324 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:48.848373 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.848374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.849017 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.849018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.849561 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:48.849563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.849909 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.849910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:48.849967 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.850411 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:48.850416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:48.852315 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
W20260806 22:40:48.852464 15015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:48.847841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:34835 (local address 127.0.186.254:36759)
0806 22:40:48.847903 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:48.847905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:48.847964 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:40:48.848034 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:48.848035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:48.848049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:48.848128 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:48.848449 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.848943 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.849650 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:48.849845 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:48.849901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:48.852411 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:34835: BlockingRecv error: recv got EOF from 127.0.186.195:34835 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:49.599311 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:49.594568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:45711)
0806 22:40:49.594721 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:49.594732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:49.594790 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:49.594906 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:49.594907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:49.595073 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:49.595124 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.595125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.595798 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.595799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.596382 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.596386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.596738 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.596739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.596789 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.597299 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:49.597304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:49.599194 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
W20260806 22:40:49.599391 15025 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:49.594607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:45711 (local address 127.0.186.254:36759)
0806 22:40:49.594721 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:49.594722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:49.594773 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:40:49.594828 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:49.594829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:49.594839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:49.594893 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:49.595173 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.595708 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.596466 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.596673 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.596724 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.599324 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:45711: BlockingRecv error: recv got EOF from 127.0.186.193:45711 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:49.747977 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:49.743557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:56127)
0806 22:40:49.743723 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:49.743736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:49.743808 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:49.743953 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:49.743954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:49.744131 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:49.744177 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.744179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.744815 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.744816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.745354 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.745356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.745685 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.745686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.745742 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.746145 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:49.746149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:49.747880 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260806 22:40:49.748040 15025 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:49.743687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:56127 (local address 127.0.186.254:36759)
0806 22:40:49.743744 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:49.743746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:49.743804 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:40:49.743882 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:49.743883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:49.743896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:49.743945 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:49.744246 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.744740 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.745431 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.745623 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.745680 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.747988 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:56127: BlockingRecv error: recv got EOF from 127.0.186.194:56127 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:49 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:49.857388 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:49.852892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:48395)
0806 22:40:49.853021 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:49.853033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:49.853110 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:49.853241 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:49.853242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:49.853426 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:49.853472 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.853474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.854119 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.854120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.854645 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:49.854646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.854977 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.854977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:49.855033 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.855594 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:49.855599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:49.857309 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
W20260806 22:40:49.857478 15025 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:49.852977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:48395 (local address 127.0.186.254:36759)
0806 22:40:49.853061 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:49.853063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:49.853088 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:40:49.853173 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:49.853174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:49.853190 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:49.853235 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:49.853546 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.854043 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.854723 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:49.854916 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:49.854971 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:49.857423 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:48395: BlockingRecv error: recv got EOF from 127.0.186.195:48395 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:50.604274 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:50.599761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:45977)
0806 22:40:50.599889 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:50.599902 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:50.599970 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:50.600102 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:50.600103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:50.600270 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:50.600320 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.600321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.600947 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.600948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.601472 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.601473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.601952 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.601953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.602018 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.602490 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:50.602494 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:50.604215 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
W20260806 22:40:50.604389 15032 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:50.599833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:45977 (local address 127.0.186.254:36759)
0806 22:40:50.599979 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:50.599981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:50.599991 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:50.600018 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:50.600019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:50.600032 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:50.600086 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:50.600375 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.600885 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.601574 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.601861 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.601950 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.604317 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:45977: BlockingRecv error: recv got EOF from 127.0.186.193:45977 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:50.756403 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:50.748454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:54063)
0806 22:40:50.748613 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:50.748625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:50.748689 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:50.748804 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:50.748805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:50.748974 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:50.749021 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.749023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.749664 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.749665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.750213 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.750215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.750509 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.750510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.750545 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.754439 (+ 3894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:50.754444 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:50.756319 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
W20260806 22:40:50.756476 15032 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:50.748534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:54063 (local address 127.0.186.254:36759)
0806 22:40:50.748611 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:50.748613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:50.748672 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:40:50.748731 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:50.748732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:50.748743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:50.748788 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:50.749101 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.749588 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.750276 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.750462 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.750507 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.756414 (+ 5907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:54063: BlockingRecv error: recv got EOF from 127.0.186.194:54063 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:50 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.195@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:50.862321 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:50.857914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.195:45399)
0806 22:40:50.858110 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:50.858122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:50.858202 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:50.858336 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:50.858338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:50.858560 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:50.858618 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.858620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.859296 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.859297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.859835 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:50.859837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.860181 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.860182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:50.860227 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.860630 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:50.860634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:50.862263 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
W20260806 22:40:50.862409 15032 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:50.857969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:45399 (local address 127.0.186.254:36759)
0806 22:40:50.858020 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:50.858022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:50.858197 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:40:50.858249 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:50.858250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:50.858266 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:50.858317 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:50.858705 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.859207 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.859924 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:50.860117 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:50.860176 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:50.862350 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.195:45399: BlockingRecv error: recv got EOF from 127.0.186.195:45399 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 22:40:51 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:51 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.193@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:51.609359 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:51.604795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.193:57275)
0806 22:40:51.604959 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:51.604982 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:51.605053 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:51.605190 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:51.605191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:51.605361 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:51.605420 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:51.605422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.606042 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.606043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:51.606594 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:51.606596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.606910 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.606910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:51.606961 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:51.607432 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:51.607437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:51.609300 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":63,"threads_started":1}
W20260806 22:40:51.609454 15037 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:51.604908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:57275 (local address 127.0.186.254:36759)
0806 22:40:51.605045 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:51.605047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:51.605056 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:51.605109 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:51.605109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:51.605120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:51.605175 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:51.605473 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.605969 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.606653 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.606834 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.606878 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:51.609391 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.193:57275: BlockingRecv error: recv got EOF from 127.0.186.193:57275 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
Aug 06 22:40:51 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:40:51 dist-test-slave-gjrz krb5kdc[13444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.186.194@KRBTEST.COM for oryx/127.0.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:40:51.761729 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:51.756914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36759 (local address 127.0.186.194:51133)
0806 22:40:51.757034 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:51.757044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:51.757108 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:51.757230 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:51.757231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:51.757406 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:51.757466 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:51.757468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.758139 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.758140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:51.758963 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:51.758967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.759483 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.759484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:51.759551 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:51.759992 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:51.759997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:51.761661 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.186.254:36759: Server oryx/127.0.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260806 22:40:51.761812 15037 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:51.757003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:51133 (local address 127.0.186.254:36759)
0806 22:40:51.757044 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:51.757046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:51.757110 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:40:51.757155 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:51.757155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:51.757167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:51.757218 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:51.757517 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.758060 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.759080 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.759400 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.759472 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:51.761753 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.186.194:51133: BlockingRecv error: recv got EOF from 127.0.186.194:51133 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:40:51.790915 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 14296
I20260806 22:40:51.807835 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 14526
I20260806 22:40:51.816457 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 14731
I20260806 22:40:51.823827 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 14198
2026-08-06T22:40:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17599 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:51Z Disabled control of system clock
I20260806 22:40:51.855773 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42627
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:40:51.937142 15044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:51.937340 15044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:51.937359 15044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:51.938472 15044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:51.938504 15044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:51.938515 15044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:51.938526 15044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:51.938537 15044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:51.939932 15044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:51.940130 15044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:51.940364 15044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:51.945478 15044 server_base.cc:1034] running on GCE node
W20260806 22:40:51.945472 15054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:51.945436 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:51.945444 15059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:51.945875 15044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:51.946202 15044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:51.947436 15044 hybrid_clock.cc:648] HybridClock initialized: now 1786056051947413 us; error 30 us; skew 500 ppm
I20260806 22:40:51.948721 15044 webserver.cc:460] Webserver started at http://127.0.186.254:42065/ using document root <none> and password file <none>
I20260806 22:40:51.948935 15044 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:51.948985 15044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:51.949131 15044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:51.950214 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "79d5a74f67954627b42d29f6ed327806"
format_stamp: "Formatted at 2026-08-06 22:40:51 on dist-test-slave-gjrz"
server_key: "be0906992b84bd605fed4b251d94a766"
server_key_iv: "b9bef8fab686ec3acbb237e230cbf7e1"
server_key_version: "encryptionkey@0"
I20260806 22:40:51.950557 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "79d5a74f67954627b42d29f6ed327806"
format_stamp: "Formatted at 2026-08-06 22:40:51 on dist-test-slave-gjrz"
server_key: "be0906992b84bd605fed4b251d94a766"
server_key_iv: "b9bef8fab686ec3acbb237e230cbf7e1"
server_key_version: "encryptionkey@0"
I20260806 22:40:51.951850 15044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:51.952560 15065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:51.952720 15044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:51.952808 15044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "79d5a74f67954627b42d29f6ed327806"
format_stamp: "Formatted at 2026-08-06 22:40:51 on dist-test-slave-gjrz"
server_key: "be0906992b84bd605fed4b251d94a766"
server_key_iv: "b9bef8fab686ec3acbb237e230cbf7e1"
server_key_version: "encryptionkey@0"
I20260806 22:40:51.952872 15044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:51.971293 15044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:51.971632 15044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:51.971768 15044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:51.975558 15044 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42627
I20260806 22:40:51.975597 15146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42627 every 8 connection(s)
I20260806 22:40:51.975937 15044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:51.976655 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:51.979447 15147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806: Bootstrap starting.
I20260806 22:40:51.980067 15147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:51.980125 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15044
I20260806 22:40:51.980221 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:51.980366 15147 log.cc:824] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:51.980387 747 external_mini_cluster.cc:1428] Setting key 94232cb301ae974a75c7610f37be8d4c
I20260806 22:40:51.981112 15147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806: No bootstrap required, opened a new log
I20260806 22:40:51.983217 15147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } }
I20260806 22:40:51.983408 15147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:51.983455 15147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d5a74f67954627b42d29f6ed327806, State: Initialized, Role: FOLLOWER
I20260806 22:40:51.983582 15147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } }
I20260806 22:40:51.983656 15147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:51.983687 15147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:51.983736 15147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:51.984431 15147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } }
I20260806 22:40:51.984540 15147 leader_election.cc:302] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d5a74f67954627b42d29f6ed327806; no voters:
I20260806 22:40:51.984691 15147 leader_election.cc:288] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:51.984738 15156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:51.984899 15156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [term 1 LEADER]: Becoming Leader. State: Replica: 79d5a74f67954627b42d29f6ed327806, State: Running, Role: LEADER
I20260806 22:40:51.984999 15156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } }
I20260806 22:40:51.985296 15147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:51.985338 15156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d5a74f67954627b42d29f6ed327806. Latest consensus state: current_term: 1 leader_uuid: "79d5a74f67954627b42d29f6ed327806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } } }
I20260806 22:40:51.985363 15157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d5a74f67954627b42d29f6ed327806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d5a74f67954627b42d29f6ed327806" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42627 } } }
I20260806 22:40:51.985507 15156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:51.985514 15157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:51.986093 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:51.980883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.0.186.254:42627)
0806 22:40:51.981111 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:51.981118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:51.981142 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:40:51.981602 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:51.981610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:51.981669 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:51.981791 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:51.982001 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.982894 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.983785 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:51.984401 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:51.984569 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:51.985236 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:51.985240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:51.985241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:51.985435 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:51.985486 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:51.985503 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:51.985755 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:51.985789 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:40:51.985881 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260806 22:40:51.986400 15161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:51.986949 15161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:51.988417 15161 catalog_manager.cc:1347] Generated new cluster ID: fd78a04e9e9b40b6bb76085867b75be7
I20260806 22:40:51.988471 15161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:52.006206 15161 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:52.006736 15161 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:52.011296 15161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d5a74f67954627b42d29f6ed327806: Generated new TSK 0
I20260806 22:40:52.011483 15161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:52.018401 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--builtin_ntp_servers=127.0.186.212:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:40:52.101230 15180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.101449 15180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.101478 15180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.102653 15180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.102695 15180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.102728 15180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:52.104099 15180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.104326 15180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.104602 15180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.110276 15192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.110378 15180 server_base.cc:1034] running on GCE node
W20260806 22:40:52.110284 15188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.110486 15187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.110683 15180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.110909 15180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.112043 15180 hybrid_clock.cc:648] HybridClock initialized: now 1786056052112029 us; error 26 us; skew 500 ppm
I20260806 22:40:52.113186 15180 webserver.cc:460] Webserver started at http://127.0.186.193:45351/ using document root <none> and password file <none>
I20260806 22:40:52.113402 15180 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.113461 15180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.113591 15180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.114657 15180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "1b1716bdac174deba5bf3ca1a6d87780"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "6687d5a99e6efa23a9e3fd2d65894831"
server_key_iv: "09611d3af7200cb206444d1e0c63ea2a"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.114991 15180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "1b1716bdac174deba5bf3ca1a6d87780"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "6687d5a99e6efa23a9e3fd2d65894831"
server_key_iv: "09611d3af7200cb206444d1e0c63ea2a"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.116218 15180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:52.116860 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.117005 15180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:52.117089 15180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "1b1716bdac174deba5bf3ca1a6d87780"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "6687d5a99e6efa23a9e3fd2d65894831"
server_key_iv: "09611d3af7200cb206444d1e0c63ea2a"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.117158 15180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.127723 15180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.127995 15180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.128127 15180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.128340 15180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.128670 15180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.128705 15180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.128732 15180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.128749 15180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.134262 15180 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:39307
I20260806 22:40:52.134299 15363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:39307 every 8 connection(s)
I20260806 22:40:52.134613 15180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:52.139562 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.135302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:32805 (local address 127.0.186.254:42627)
0806 22:40:52.135357 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.135359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.135573 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:40:52.136106 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.136107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.136119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.136180 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.136825 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.137326 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.138264 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.138463 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.138518 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.139293 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.139295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.139295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.139300 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.139317 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.139321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.139367 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.139447 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.139475 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:40:52.139760 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.135253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42627 (local address 127.0.186.193:32805)
0806 22:40:52.135462 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.135495 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.135985 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.136255 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.136259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.136270 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.136739 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.136743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.137412 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.137413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.138183 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.138185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.138532 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.138533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.138662 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.139136 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.139137 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.139166 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.139188 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.139376 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.139382 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.139385 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.139446 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.139551 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260806 22:40:52.140230 15364 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42627
I20260806 22:40:52.140312 15364 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.140460 15364 heartbeater.cc:511] Master 127.0.186.254:42627 requested a full tablet report, sending...
I20260806 22:40:52.140923 15091 ts_manager.cc:194] Registered new tserver with Master: 1b1716bdac174deba5bf3ca1a6d87780 (127.0.186.193:39307)
I20260806 22:40:52.141489 15091 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.193:32805
I20260806 22:40:52.144389 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15180
I20260806 22:40:52.144472 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:52.144647 747 external_mini_cluster.cc:1428] Setting key 4cadff83b444d00983c9d7074fa3621b
I20260806 22:40:52.145493 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--builtin_ntp_servers=127.0.186.212:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:40:52.226145 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.226336 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.226356 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.227507 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.227546 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.227571 15370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:52.228909 15370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.229130 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.229393 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.234174 15384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.234251 15381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.234252 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.234292 15370 server_base.cc:1034] running on GCE node
I20260806 22:40:52.234622 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.234841 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.235980 15370 hybrid_clock.cc:648] HybridClock initialized: now 1786056052235964 us; error 29 us; skew 500 ppm
I20260806 22:40:52.237149 15370 webserver.cc:460] Webserver started at http://127.0.186.194:34579/ using document root <none> and password file <none>
I20260806 22:40:52.237361 15370 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.237408 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.237555 15370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.238569 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "fc304fd4f4bd401ab68b36b61ad3172b"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a59813083c93cc1286e222598e9b2533"
server_key_iv: "3df2f941a4536ca9fc00ba13eeb49c7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.238894 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "fc304fd4f4bd401ab68b36b61ad3172b"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a59813083c93cc1286e222598e9b2533"
server_key_iv: "3df2f941a4536ca9fc00ba13eeb49c7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.240185 15370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:52.240913 15395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.241066 15370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:52.241150 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "fc304fd4f4bd401ab68b36b61ad3172b"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a59813083c93cc1286e222598e9b2533"
server_key_iv: "3df2f941a4536ca9fc00ba13eeb49c7f"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.241217 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.270622 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.270960 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.271158 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.271441 15370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.271822 15370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.271917 15370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.271981 15370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.272010 15370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.282320 15370 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:35235
I20260806 22:40:52.282370 15560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:35235 every 8 connection(s)
I20260806 22:40:52.282862 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:52.290364 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.283701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:36275 (local address 127.0.186.254:42627)
0806 22:40:52.283758 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.283760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.284019 (+ 259us) server_negotiation.cc:407] Connection header received
0806 22:40:52.284743 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.284745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.284758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.284816 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.285739 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.286346 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.287906 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.288270 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.288322 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.290082 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.290084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.290084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.290091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.290112 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.290117 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.290182 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.290235 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.290267 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:52.290513 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.283655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42627 (local address 127.0.186.194:36275)
0806 22:40:52.283878 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.283925 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.284608 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.284914 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.284918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.284929 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.285637 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.285646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.286444 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.286445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.287803 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.287807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.288446 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.288447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.288646 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.289919 (+ 1273us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.289920 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.289954 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.289979 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.290181 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.290184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.290186 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.290232 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.290313 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260806 22:40:52.291396 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15370
I20260806 22:40:52.291494 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:52.291679 747 external_mini_cluster.cc:1428] Setting key 8fb2392216b9e638acc80873a4b10f19
I20260806 22:40:52.291783 15561 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42627
I20260806 22:40:52.291883 15561 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.292076 15561 heartbeater.cc:511] Master 127.0.186.254:42627 requested a full tablet report, sending...
I20260806 22:40:52.292444 15091 ts_manager.cc:194] Registered new tserver with Master: fc304fd4f4bd401ab68b36b61ad3172b (127.0.186.194:35235)
I20260806 22:40:52.292665 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--builtin_ntp_servers=127.0.186.212:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:40:52.292888 15091 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.194:36275
W20260806 22:40:52.377894 15564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.378094 15564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.378123 15564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.379243 15564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.379310 15564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.379344 15564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:52.380666 15564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.380892 15564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.381155 15564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.385833 15573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.385865 15575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.385871 15578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.386099 15564 server_base.cc:1034] running on GCE node
I20260806 22:40:52.386365 15564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.386600 15564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.387737 15564 hybrid_clock.cc:648] HybridClock initialized: now 1786056052387719 us; error 31 us; skew 500 ppm
I20260806 22:40:52.388898 15564 webserver.cc:460] Webserver started at http://127.0.186.195:34917/ using document root <none> and password file <none>
I20260806 22:40:52.389109 15564 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.389158 15564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.389281 15564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.390416 15564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "88a46ce3e3804e838f01893739ab00ba"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4cd29c5ac6e263afa2f388d8b14f601b"
server_key_iv: "7527a237c5c2c76413e365f0aeb7f548"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.390755 15564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "88a46ce3e3804e838f01893739ab00ba"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4cd29c5ac6e263afa2f388d8b14f601b"
server_key_iv: "7527a237c5c2c76413e365f0aeb7f548"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.392063 15564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:52.392714 15585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.392861 15564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:52.392946 15564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "88a46ce3e3804e838f01893739ab00ba"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4cd29c5ac6e263afa2f388d8b14f601b"
server_key_iv: "7527a237c5c2c76413e365f0aeb7f548"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.393004 15564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.404067 15564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.404330 15564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.404455 15564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.404675 15564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.404973 15564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.405007 15564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.405036 15564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.405058 15564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.410451 15564 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35241
I20260806 22:40:52.410492 15748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35241 every 8 connection(s)
I20260806 22:40:52.410799 15564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:52.415488 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.411517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:44275 (local address 127.0.186.254:42627)
0806 22:40:52.411584 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.411586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.411805 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:40:52.412271 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.412273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.412282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.412335 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.412931 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.413407 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.414328 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.414500 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.414540 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.415308 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.415308 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.415309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.415313 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.415323 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.415326 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.415359 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.415415 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.415432 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:40:52.415688 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.411479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42627 (local address 127.0.186.195:44275)
0806 22:40:52.411691 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.411731 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.412198 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.412398 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.412401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.412409 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.412871 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.412875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.413475 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.413475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.414268 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:52.414270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:52.414556 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:52.414556 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:52.414659 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:52.415163 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.415164 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.415192 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.415215 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.415363 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.415366 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.415368 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.415414 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.415490 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260806 22:40:52.416119 15749 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42627
I20260806 22:40:52.416208 15749 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.416352 15749 heartbeater.cc:511] Master 127.0.186.254:42627 requested a full tablet report, sending...
I20260806 22:40:52.416697 15091 ts_manager.cc:194] Registered new tserver with Master: 88a46ce3e3804e838f01893739ab00ba (127.0.186.195:35241)
I20260806 22:40:52.417029 15091 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.195:44275
I20260806 22:40:52.418749 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15564
I20260806 22:40:52.418818 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:52.418964 747 external_mini_cluster.cc:1428] Setting key 66f8b670ecc8498588d9a2f29b654a31
I20260806 22:40:52.419770 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:40:52.421518 15155 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:52.421034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.0.186.254:42627)
0806 22:40:52.421083 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.421085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.421205 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:40:52.421272 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.421273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.421282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.421333 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.421460 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39034: BlockingRecv error: recv got EOF from 127.0.0.1:39034 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:40:52.421554 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15180
I20260806 22:40:52.439190 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15370
I20260806 22:40:52.447326 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15564
I20260806 22:40:52.457474 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15044
2026-08-06T22:40:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (636 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:40:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:52Z Disabled control of system clock
I20260806 22:40:52.492584 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42269
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:38733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:40:52.577044 15764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.577251 15764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.577272 15764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.578368 15764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:52.578402 15764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.578414 15764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.578425 15764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:52.578442 15764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:52.579778 15764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42269
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.579972 15764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.580204 15764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.584750 15770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.584867 15764 server_base.cc:1034] running on GCE node
W20260806 22:40:52.584759 15773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.584980 15769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.585265 15764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.585562 15764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.586719 15764 hybrid_clock.cc:648] HybridClock initialized: now 1786056052586704 us; error 37 us; skew 500 ppm
I20260806 22:40:52.587986 15764 webserver.cc:460] Webserver started at http://127.0.186.254:33159/ using document root <none> and password file <none>
I20260806 22:40:52.588197 15764 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.588243 15764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.588366 15764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.589398 15764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "df7c77151512489c92d514bca2e25e93"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3d9518fcb06d74d1d23b44a46afc6068"
server_key_iv: "8a1bf2f0271fed9f22c4971e131ed269"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.589735 15764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "df7c77151512489c92d514bca2e25e93"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3d9518fcb06d74d1d23b44a46afc6068"
server_key_iv: "8a1bf2f0271fed9f22c4971e131ed269"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.590912 15764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:52.591624 15782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.591759 15764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.591838 15764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "df7c77151512489c92d514bca2e25e93"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3d9518fcb06d74d1d23b44a46afc6068"
server_key_iv: "8a1bf2f0271fed9f22c4971e131ed269"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.591902 15764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.595117 15764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.595391 15764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.595507 15764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.599170 15764 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42269
I20260806 22:40:52.599201 15850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42269 every 8 connection(s)
I20260806 22:40:52.599514 15764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:52.600152 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:52.602864 15851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93: Bootstrap starting.
I20260806 22:40:52.603453 15851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:52.603724 15851 log.cc:824] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:52.604344 15851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93: No bootstrap required, opened a new log
I20260806 22:40:52.605863 15851 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } }
I20260806 22:40:52.605993 15851 raft_consensus.cc:374] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:52.606026 15851 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7c77151512489c92d514bca2e25e93, State: Initialized, Role: FOLLOWER
I20260806 22:40:52.606163 15851 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } }
I20260806 22:40:52.606225 15851 raft_consensus.cc:388] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:52.606245 15851 raft_consensus.cc:482] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:52.606276 15851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:52.606549 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15764
I20260806 22:40:52.606647 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:52.606837 747 external_mini_cluster.cc:1428] Setting key 17bf32d69a475efbf8116e8e40d64a42
I20260806 22:40:52.606927 15851 raft_consensus.cc:504] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } }
I20260806 22:40:52.607033 15851 leader_election.cc:302] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df7c77151512489c92d514bca2e25e93; no voters:
I20260806 22:40:52.607183 15851 leader_election.cc:288] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:52.607216 15854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:52.607537 15854 raft_consensus.cc:686] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [term 1 LEADER]: Becoming Leader. State: Replica: df7c77151512489c92d514bca2e25e93, State: Running, Role: LEADER
I20260806 22:40:52.607558 15851 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:52.607693 15854 consensus_queue.cc:237] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } }
I20260806 22:40:52.608690 15861 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df7c77151512489c92d514bca2e25e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } } }
I20260806 22:40:52.608774 15861 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:52.609118 15878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:52.609107 15859 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df7c77151512489c92d514bca2e25e93. Latest consensus state: current_term: 1 leader_uuid: "df7c77151512489c92d514bca2e25e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7c77151512489c92d514bca2e25e93" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42269 } } }
I20260806 22:40:52.609400 15859 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:52.609761 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.607378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.0.186.254:42269)
0806 22:40:52.607751 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.607760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.607795 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:40:52.607899 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.607904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.607935 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.608073 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.609166 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.609169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.609170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.609174 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.609205 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.609218 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.609366 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.609522 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.609589 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":68,"threads_started":1}
I20260806 22:40:52.609838 15878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:52.611353 15878 catalog_manager.cc:1347] Generated new cluster ID: f13c86fc16e4485db9e315a312c1a7af
I20260806 22:40:52.611398 15878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:52.622493 15878 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:52.622632 15878 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:52.628013 15878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df7c77151512489c92d514bca2e25e93: Generated new TSK 0
I20260806 22:40:52.628221 15878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:52.636384 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--builtin_ntp_servers=127.0.186.212:38733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:40:52.721401 15891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.721604 15891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.721634 15891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.722798 15891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.722837 15891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.722872 15891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:52.724215 15891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.724433 15891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.724694 15891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.729393 15896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.729516 15897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.729394 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.730285 15891 server_base.cc:1034] running on GCE node
I20260806 22:40:52.730495 15891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.730751 15891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.731909 15891 hybrid_clock.cc:648] HybridClock initialized: now 1786056052731898 us; error 32 us; skew 500 ppm
I20260806 22:40:52.733214 15891 webserver.cc:460] Webserver started at http://127.0.186.193:37981/ using document root <none> and password file <none>
I20260806 22:40:52.733434 15891 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.733479 15891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.733615 15891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.734642 15891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "8aecd26875cd4d3eb60017cc1ad57d17"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3a5e8d42517444a646f28a83c499c939"
server_key_iv: "4462581e128f8207b375804da69b5ed9"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.734972 15891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "8aecd26875cd4d3eb60017cc1ad57d17"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3a5e8d42517444a646f28a83c499c939"
server_key_iv: "4462581e128f8207b375804da69b5ed9"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.736215 15891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:52.736922 15908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.737099 15891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:52.737182 15891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "8aecd26875cd4d3eb60017cc1ad57d17"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "3a5e8d42517444a646f28a83c499c939"
server_key_iv: "4462581e128f8207b375804da69b5ed9"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.737251 15891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.740492 15891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.740736 15891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.740854 15891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.741046 15891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.741325 15891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.741358 15891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.741384 15891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.741401 15891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.746711 15891 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:42523
I20260806 22:40:52.746749 16064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:42523 every 8 connection(s)
I20260806 22:40:52.747026 15891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:52.749637 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.747673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:35471 (local address 127.0.186.254:42269)
0806 22:40:52.747749 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.747751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.747923 (+ 172us) server_negotiation.cc:407] Connection header received
0806 22:40:52.748330 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.748331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.748337 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.748381 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.749424 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.749426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.749426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.749431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.749463 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.749468 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.749523 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.749574 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.749586 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:40:52.749797 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.747651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42269 (local address 127.0.186.193:35471)
0806 22:40:52.747816 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.747847 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.748260 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.748439 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.748442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.748450 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.749274 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.749274 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.749301 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.749328 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.749520 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.749523 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.749525 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.749579 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.749626 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 22:40:52.750245 16065 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42269
I20260806 22:40:52.750327 16065 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.750442 16065 heartbeater.cc:511] Master 127.0.186.254:42269 requested a full tablet report, sending...
I20260806 22:40:52.750896 15803 ts_manager.cc:194] Registered new tserver with Master: 8aecd26875cd4d3eb60017cc1ad57d17 (127.0.186.193:42523)
I20260806 22:40:52.752306 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 15891
I20260806 22:40:52.752391 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:52.752528 747 external_mini_cluster.cc:1428] Setting key 1074a7687b5e6e8c6cd8a0a9eeb3e313
I20260806 22:40:52.753276 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--builtin_ntp_servers=127.0.186.212:38733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:40:52.833660 16068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.833871 16068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.833900 16068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.835044 16068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.835083 16068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.835115 16068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:52.836450 16068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.836676 16068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.836939 16068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.841766 16077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.841885 16078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.841835 16080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.842105 16068 server_base.cc:1034] running on GCE node
I20260806 22:40:52.842301 16068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.842546 16068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.843700 16068 hybrid_clock.cc:648] HybridClock initialized: now 1786056052843685 us; error 28 us; skew 500 ppm
I20260806 22:40:52.844837 16068 webserver.cc:460] Webserver started at http://127.0.186.194:39247/ using document root <none> and password file <none>
I20260806 22:40:52.845052 16068 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.845098 16068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.845229 16068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.846263 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "6e560b632cd7431780fcb3b41b3f7914"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4955c4ddd9d713c6834121f2aca92d5e"
server_key_iv: "4cd69fbeb8184e9ee10cb6fa0e96dc36"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.846603 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "6e560b632cd7431780fcb3b41b3f7914"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4955c4ddd9d713c6834121f2aca92d5e"
server_key_iv: "4cd69fbeb8184e9ee10cb6fa0e96dc36"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.847929 16068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:52.848694 16090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.848882 16068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:52.848975 16068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "6e560b632cd7431780fcb3b41b3f7914"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "4955c4ddd9d713c6834121f2aca92d5e"
server_key_iv: "4cd69fbeb8184e9ee10cb6fa0e96dc36"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.849076 16068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.852396 16068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.852666 16068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.852797 16068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.852986 16068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.853279 16068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.853312 16068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.853338 16068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.853363 16068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.858779 16068 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:41917
I20260806 22:40:52.858840 16260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:41917 every 8 connection(s)
I20260806 22:40:52.859127 16068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:52.862040 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.859881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:59701 (local address 127.0.186.254:42269)
0806 22:40:52.859939 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.859941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.860175 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:40:52.860646 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.860648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.860654 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.860716 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.861796 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.861798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.861798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.861803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.861820 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.861824 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.861897 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.861950 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.861962 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:40:52.862161 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.859852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42269 (local address 127.0.186.194:59701)
0806 22:40:52.860058 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.860093 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.860521 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.860776 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.860779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.860787 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.861636 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.861636 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.861666 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.861695 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.861897 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.861899 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.861901 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.861950 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.861997 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
I20260806 22:40:52.862598 16261 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42269
I20260806 22:40:52.862689 16261 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.862828 16261 heartbeater.cc:511] Master 127.0.186.254:42269 requested a full tablet report, sending...
I20260806 22:40:52.863166 15803 ts_manager.cc:194] Registered new tserver with Master: 6e560b632cd7431780fcb3b41b3f7914 (127.0.186.194:41917)
I20260806 22:40:52.868207 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 16068
I20260806 22:40:52.868294 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:52.868454 747 external_mini_cluster.cc:1428] Setting key 637feef7f3fd39eca96b0bd886830774
I20260806 22:40:52.869339 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--builtin_ntp_servers=127.0.186.212:38733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:40:52.951031 16265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:52.951233 16265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:52.951256 16265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:52.952512 16265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:52.952560 16265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:52.952590 16265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:52.953980 16265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:52.954205 16265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:52.954455 16265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:52.959106 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:52.959106 16276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.959157 16265 server_base.cc:1034] running on GCE node
W20260806 22:40:52.959137 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:52.959538 16265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:52.959761 16265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:52.960901 16265 hybrid_clock.cc:648] HybridClock initialized: now 1786056052960881 us; error 31 us; skew 500 ppm
I20260806 22:40:52.962090 16265 webserver.cc:460] Webserver started at http://127.0.186.195:43475/ using document root <none> and password file <none>
I20260806 22:40:52.962311 16265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:52.962365 16265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:52.962517 16265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:52.963601 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "ba8e1ad92b3940238faaae6bba59e788"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a7f21576e0328bf017dc27c8774f7ff8"
server_key_iv: "fe840ebff1fd702dc251e31c03cd1fb2"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.963943 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "ba8e1ad92b3940238faaae6bba59e788"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a7f21576e0328bf017dc27c8774f7ff8"
server_key_iv: "fe840ebff1fd702dc251e31c03cd1fb2"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.965240 16265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:52.965903 16283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.966069 16265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:52.966150 16265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "ba8e1ad92b3940238faaae6bba59e788"
format_stamp: "Formatted at 2026-08-06 22:40:52 on dist-test-slave-gjrz"
server_key: "a7f21576e0328bf017dc27c8774f7ff8"
server_key_iv: "fe840ebff1fd702dc251e31c03cd1fb2"
server_key_version: "encryptionkey@0"
I20260806 22:40:52.966215 16265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:52.969530 16265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:52.969769 16265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:52.969882 16265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:52.970062 16265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:52.970342 16265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:52.970374 16265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.970402 16265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:52.970425 16265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:52.975859 16265 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:41533
I20260806 22:40:52.975899 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:41533 every 8 connection(s)
I20260806 22:40:52.976189 16265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:52.978966 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.976863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:34339 (local address 127.0.186.254:42269)
0806 22:40:52.976925 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.976926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.977135 (+ 209us) server_negotiation.cc:407] Connection header received
0806 22:40:52.977606 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.977608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.977614 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.977662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.978734 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:52.978736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:52.978736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:52.978741 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:52.978757 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:52.978762 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:52.978812 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:52.978880 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:40:52.978895 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:40:52.979144 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:52.976830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42269 (local address 127.0.186.195:34339)
0806 22:40:52.977022 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:52.977054 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:52.977505 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:52.977753 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:52.977757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:52.977767 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:52.978592 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:40:52.978592 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:52.978620 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:40:52.978648 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:52.978814 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:52.978819 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:52.978822 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:52.978890 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:40:52.978950 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260806 22:40:52.979641 16445 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42269
I20260806 22:40:52.979729 16445 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:52.979861 16445 heartbeater.cc:511] Master 127.0.186.254:42269 requested a full tablet report, sending...
I20260806 22:40:52.980214 15803 ts_manager.cc:194] Registered new tserver with Master: ba8e1ad92b3940238faaae6bba59e788 (127.0.186.195:41533)
I20260806 22:40:52.984613 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 16265
I20260806 22:40:52.984684 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:52.984848 747 external_mini_cluster.cc:1428] Setting key 8dd83f5cca18a1da3df60de25d6555d2
I20260806 22:40:52.985633 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:40:52.987380 15860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:52.986993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.0.186.254:42269)
0806 22:40:52.987062 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:52.987064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:52.987102 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:40:52.987152 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:52.987153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:52.987157 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:52.987201 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:52.987321 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38196: BlockingRecv error: recv got EOF from 127.0.0.1:38196 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:40:52.987412 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15891
I20260806 22:40:52.999963 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 16068
I20260806 22:40:53.009145 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 16265
I20260806 22:40:53.020632 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 15764
2026-08-06T22:40:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (564 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:53 dist-test-slave-gjrz krb5kdc[16458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:53 dist-test-slave-gjrz krb5kdc[16458](info): set up 2 sockets
Aug 06 22:40:53 dist-test-slave-gjrz krb5kdc[16458](info): commencing operation
krb5kdc: starting...
W20260806 22:40:54.163192 747 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:40:54.278227 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42703
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:54.360566 16473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:54.360761 16473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:54.360790 16473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:54.361907 16473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:54.361945 16473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:54.361963 16473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:54.361979 16473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:54.361994 16473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:54.363533 16473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:54.363768 16473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:54.363966 16473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:54.368505 16489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.368489 16485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:54.368644 16473 server_base.cc:1034] running on GCE node
W20260806 22:40:54.368469 16484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:54.368991 16473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:54.369262 16473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:54.370417 16473 hybrid_clock.cc:648] HybridClock initialized: now 1786056054370393 us; error 34 us; skew 500 ppm
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:54.372918 16473 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:40:54.373364 16473 webserver.cc:460] Webserver started at http://127.0.186.254:35913/ using document root <none> and password file <none>
I20260806 22:40:54.373579 16473 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:54.373629 16473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:54.373751 16473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:54.374774 16473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "49204da3b22d4b05a9444d86ce367835"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "02bb05ccd1242efcfd752a20943bdc47"
server_key_iv: "2ae26b7a569733892800ed456cd9c715"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.375110 16473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "49204da3b22d4b05a9444d86ce367835"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "02bb05ccd1242efcfd752a20943bdc47"
server_key_iv: "2ae26b7a569733892800ed456cd9c715"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.376427 16473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:54.377152 16502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.377301 16473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:54.377388 16473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "49204da3b22d4b05a9444d86ce367835"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "02bb05ccd1242efcfd752a20943bdc47"
server_key_iv: "2ae26b7a569733892800ed456cd9c715"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.377466 16473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:54.392304 16473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:54.393990 16473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:54.394124 16473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:54.398021 16473 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42703
I20260806 22:40:54.398052 16578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42703 every 8 connection(s)
I20260806 22:40:54.398409 16473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:54.399492 16579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:54.401778 16579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: Bootstrap starting.
I20260806 22:40:54.402330 16579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:54.402585 16579 log.cc:824] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:54.403144 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 16473
I20260806 22:40:54.403230 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:40:54.403247 16579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: No bootstrap required, opened a new log
I20260806 22:40:54.403420 747 external_mini_cluster.cc:1428] Setting key 28912fe6fb0e04d6d75f000abe11f66d
I20260806 22:40:54.405242 16579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } }
I20260806 22:40:54.405380 16579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:54.405412 16579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49204da3b22d4b05a9444d86ce367835, State: Initialized, Role: FOLLOWER
I20260806 22:40:54.405555 16579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } }
I20260806 22:40:54.405632 16579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:54.405665 16579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:54.405712 16579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:54.406545 16579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } }
I20260806 22:40:54.406644 16579 leader_election.cc:302] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49204da3b22d4b05a9444d86ce367835; no voters:
I20260806 22:40:54.406796 16579 leader_election.cc:288] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:54.406839 16586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:54.406991 16586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [term 1 LEADER]: Becoming Leader. State: Replica: 49204da3b22d4b05a9444d86ce367835, State: Running, Role: LEADER
I20260806 22:40:54.407053 16579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:54.407076 16586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } }
I20260806 22:40:54.407401 16588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49204da3b22d4b05a9444d86ce367835. Latest consensus state: current_term: 1 leader_uuid: "49204da3b22d4b05a9444d86ce367835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } } }
I20260806 22:40:54.407483 16588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:54.407688 16593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:54.408277 16593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:54.408865 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49204da3b22d4b05a9444d86ce367835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49204da3b22d4b05a9444d86ce367835" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42703 } } }
I20260806 22:40:54.408964 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:54.410089 16593 catalog_manager.cc:1347] Generated new cluster ID: f98e04758f3d4128a1494af24a2be2fc
I20260806 22:40:54.410140 16593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:54.414346 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.403929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.0.186.254:42703)
0806 22:40:54.404134 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.404144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.404176 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:40:54.405229 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.405238 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.405291 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.405434 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:54.406114 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.407201 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.408310 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.408943 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.409056 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.411446 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:54.411450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:54.411450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:54.411462 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:54.412981 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.413297 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.413300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.413300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.413411 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.413590 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.413591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.413591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.413814 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:54.413969 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.414039 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.414172 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260806 22:40:54.432940 16593 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:54.433472 16593 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:54.437680 16593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: Generated new TSK 0
I20260806 22:40:54.437848 16593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:40:54.467603 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:54.568007 16609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:54.568214 16609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:54.568246 16609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:54.569360 16609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:54.569401 16609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:54.569434 16609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:40:54.570851 16609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:54.571069 16609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:54.571303 16609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:54.576009 16619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.576057 16624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.576030 16618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:54.576226 16609 server_base.cc:1034] running on GCE node
I20260806 22:40:54.576440 16609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:54.576642 16609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:54.577772 16609 hybrid_clock.cc:648] HybridClock initialized: now 1786056054577761 us; error 30 us; skew 500 ppm
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:54.580307 16609 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:40:54.580734 16609 webserver.cc:460] Webserver started at http://127.0.186.193:38207/ using document root <none> and password file <none>
I20260806 22:40:54.580924 16609 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:54.580983 16609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:54.581108 16609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:54.582145 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "fc6642c5bb2e434fae99a2d9132339a7"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d167fd8309600aef4bb3f9aaefc01efb"
server_key_iv: "585ff1693baec2153c84ef47ff39845b"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.582475 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "fc6642c5bb2e434fae99a2d9132339a7"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d167fd8309600aef4bb3f9aaefc01efb"
server_key_iv: "585ff1693baec2153c84ef47ff39845b"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.583781 16609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:54.584453 16633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.584600 16609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:54.584685 16609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "fc6642c5bb2e434fae99a2d9132339a7"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d167fd8309600aef4bb3f9aaefc01efb"
server_key_iv: "585ff1693baec2153c84ef47ff39845b"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.584743 16609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:54.593601 16609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:54.595628 16609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:54.595758 16609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:54.595954 16609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:54.596237 16609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:54.596271 16609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.596297 16609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:54.596314 16609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.601742 16609 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:45345
I20260806 22:40:54.601780 16804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:45345 every 8 connection(s)
I20260806 22:40:54.602077 16609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:54.603317 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 16609
I20260806 22:40:54.603410 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:54.603610 747 external_mini_cluster.cc:1428] Setting key fb4dd7a9234a20c56199d380c5ea34d1
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:54.611618 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.602722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:49335 (local address 127.0.186.254:42703)
0806 22:40:54.602777 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.602780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.602941 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:40:54.603491 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.603493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.603511 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.603571 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:54.604557 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.605058 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.606255 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.606479 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.606540 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.608894 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:54.608896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:54.608896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:54.608903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:54.610343 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.610714 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.610715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.610716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.610782 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.610975 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.610976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.610976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.611110 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:54.611185 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.611429 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.611522 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:40:54.611661 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.602678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42703 (local address 127.0.186.193:49335)
0806 22:40:54.602838 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.602870 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.603371 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.603636 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.603640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.604236 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:54.604472 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.604478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.605142 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.605143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.606165 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.606168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.606558 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.606559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.606694 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.607148 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:54.607156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:54.608806 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:54.610430 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.610436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.610437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.610647 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.610841 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.610841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.610842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.610923 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.611170 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:54.611172 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:54.611299 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.611389 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.611503 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260806 22:40:54.612125 16807 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42703
I20260806 22:40:54.612210 16807 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:54.612367 16807 heartbeater.cc:511] Master 127.0.186.254:42703 requested a full tablet report, sending...
I20260806 22:40:54.612829 16526 ts_manager.cc:194] Registered new tserver with Master: fc6642c5bb2e434fae99a2d9132339a7 (127.0.186.193:45345)
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
I20260806 22:40:54.613396 16526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:49335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:40:54.627759 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:54.707094 16815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:54.707320 16815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:54.707356 16815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:54.708513 16815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:54.708551 16815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:54.708586 16815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:40:54.710004 16815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:54.710219 16815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:54.710422 16815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:54.715070 16826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.715080 16828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.715286 16832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:54.715427 16815 server_base.cc:1034] running on GCE node
I20260806 22:40:54.715592 16815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:54.715814 16815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:54.716956 16815 hybrid_clock.cc:648] HybridClock initialized: now 1786056054716939 us; error 26 us; skew 500 ppm
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:54.719224 16815 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:40:54.719623 16815 webserver.cc:460] Webserver started at http://127.0.186.194:44175/ using document root <none> and password file <none>
I20260806 22:40:54.719786 16815 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:54.719822 16815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:54.719909 16815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:54.720849 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "50878e5494aa451bb32a6e373e76dd86"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "9dd950c1d4270ee21601ab21fe3681e8"
server_key_iv: "11b27378efe3bf50afc8c2eac721ea3f"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.721138 16815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "50878e5494aa451bb32a6e373e76dd86"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "9dd950c1d4270ee21601ab21fe3681e8"
server_key_iv: "11b27378efe3bf50afc8c2eac721ea3f"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.722266 16815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:40:54.722863 16840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.723006 16815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.723078 16815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "50878e5494aa451bb32a6e373e76dd86"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "9dd950c1d4270ee21601ab21fe3681e8"
server_key_iv: "11b27378efe3bf50afc8c2eac721ea3f"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.723146 16815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:54.735896 16815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:54.737709 16815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:54.737825 16815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:54.738025 16815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:54.738306 16815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:54.738339 16815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.738358 16815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:54.738369 16815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.743793 16815 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43609
I20260806 22:40:54.743834 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43609 every 8 connection(s)
I20260806 22:40:54.744117 16815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:54.752117 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 16815
I20260806 22:40:54.752213 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:54.752418 747 external_mini_cluster.cc:1428] Setting key b7f37aebfe0d24c83c2b810bd41cabc2
I20260806 22:40:54.752511 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.744735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:46175 (local address 127.0.186.254:42703)
0806 22:40:54.744818 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.744820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.744971 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:40:54.745396 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.745397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.745407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.745463 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:54.746297 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.746786 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.747774 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.747975 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.748036 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.749994 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:54.749996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:54.749996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:54.750003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:54.751289 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.751616 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.751617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.751618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.751670 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.751844 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.751844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.751845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.752000 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:54.752081 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.752327 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.752414 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:40:54.752622 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.744712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42703 (local address 127.0.186.194:46175)
0806 22:40:54.744851 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.744883 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.745321 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.745514 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.745517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.746037 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:54.746221 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.746226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.746850 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.746851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.747690 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.747693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.748036 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.748037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.748140 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.748531 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:54.748538 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:54.749885 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:54.751351 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.751355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.751357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.751553 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.751718 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.751719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.751719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.751802 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.752066 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:54.752068 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:54.752180 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.752284 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.752410 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260806 22:40:54.753029 17010 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42703
I20260806 22:40:54.753123 17010 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:54.753310 17010 heartbeater.cc:511] Master 127.0.186.254:42703 requested a full tablet report, sending...
I20260806 22:40:54.753703 16526 ts_manager.cc:194] Registered new tserver with Master: 50878e5494aa451bb32a6e373e76dd86 (127.0.186.194:43609)
I20260806 22:40:54.754055 16526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:46175
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:40:54.777406 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:54.857806 17016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:54.858001 17016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:54.858024 17016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:54.859134 17016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:54.859176 17016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:54.859201 17016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:40:54.860625 17016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:40:54.860828 17016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:54.861042 17016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:54.865803 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.865796 17023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:54.865796 17026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:54.866037 17016 server_base.cc:1034] running on GCE node
I20260806 22:40:54.866230 17016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:54.866428 17016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:54.867568 17016 hybrid_clock.cc:648] HybridClock initialized: now 1786056054867554 us; error 31 us; skew 500 ppm
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:54.870082 17016 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:40:54.870489 17016 webserver.cc:460] Webserver started at http://127.0.186.195:39369/ using document root <none> and password file <none>
I20260806 22:40:54.870702 17016 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:54.870750 17016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:54.870874 17016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:54.871958 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "43c8b3cbd278475292fb32444f069fe2"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d6a2b7c4a2dbeda09ac8d79a8c76bee5"
server_key_iv: "483e2dd494450958a3c43219e3b119f9"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.872287 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "43c8b3cbd278475292fb32444f069fe2"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d6a2b7c4a2dbeda09ac8d79a8c76bee5"
server_key_iv: "483e2dd494450958a3c43219e3b119f9"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.873512 17016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:54.874276 17035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.874469 17016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:54.874559 17016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "43c8b3cbd278475292fb32444f069fe2"
format_stamp: "Formatted at 2026-08-06 22:40:54 on dist-test-slave-gjrz"
server_key: "d6a2b7c4a2dbeda09ac8d79a8c76bee5"
server_key_iv: "483e2dd494450958a3c43219e3b119f9"
server_key_version: "encryptionkey@0"
I20260806 22:40:54.874619 17016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:54.887861 17016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:54.889928 17016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:54.890059 17016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:54.890269 17016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:54.890559 17016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:54.890592 17016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.890620 17016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:54.890643 17016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:54.896081 17016 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:46195
I20260806 22:40:54.896135 17192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:46195 every 8 connection(s)
I20260806 22:40:54.896407 17016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:54 dist-test-slave-gjrz krb5kdc[16458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:40:54.902328 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 17016
I20260806 22:40:54.902441 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:54.902649 747 external_mini_cluster.cc:1428] Setting key fc889dee88f1c78ab0e2fdb0a65c94cf
I20260806 22:40:54.904935 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.897075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:56275 (local address 127.0.186.254:42703)
0806 22:40:54.897126 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.897128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.897312 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:40:54.897796 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.897798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.897810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.897861 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:54.898758 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.899249 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.900239 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.900428 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.900483 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.902370 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:54.902372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:54.902373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:54.902381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:54.903822 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.904174 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.904175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.904175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.904228 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.904406 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.904407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.904407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.904515 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:54.904562 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.904801 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.904873 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:40:54.905021 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.897051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42703 (local address 127.0.186.195:56275)
0806 22:40:54.897205 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.897238 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.897706 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.897930 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.897935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.898484 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:54.898663 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.898668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.899367 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.899368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.900151 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.900153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.900495 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.900496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.900608 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.900976 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:54.900983 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:54.902278 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:54.903886 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.903892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.903894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.904112 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.904278 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:54.904279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:54.904279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:54.904365 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:54.904560 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:54.904564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:54.904668 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.904745 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.904860 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 22:40:54.905427 17194 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42703
I20260806 22:40:54.905521 17194 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:54.905710 17194 heartbeater.cc:511] Master 127.0.186.254:42703 requested a full tablet report, sending...
I20260806 22:40:54.906091 16526 ts_manager.cc:194] Registered new tserver with Master: 43c8b3cbd278475292fb32444f069fe2 (127.0.186.195:46195)
I20260806 22:40:54.906438 16526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:56275
I20260806 22:40:54.914558 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:54.920694 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.915809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.0.186.254:42703)
0806 22:40:54.915846 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.915847 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.915929 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:40:54.915983 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.915984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.915991 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.916036 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:54.916499 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.916938 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.917619 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.917783 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.917823 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.918751 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:54.918752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:54.918752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:54.918758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:54.919819 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.920153 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.920154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.920154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.920200 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:54.920327 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:54.920328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:54.920328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:54.920445 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:54.920512 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.920586 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.920638 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:40:54.922112 16526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:54.923666 16526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:54.929728 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:46195 (local address 127.0.0.1:40030)
0806 22:40:54.926239 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.926249 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.926297 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.926520 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.926520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.926522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:54.926567 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.926568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.927759 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.927760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.928651 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.928653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.929444 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.929444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.929526 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.929540 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.929609 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.929658 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":47,"threads_started":1}
I20260806 22:40:54.929741 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.0.186.195:46195)
0806 22:40:54.926064 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.926069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.926349 (+ 280us) server_negotiation.cc:407] Connection header received
0806 22:40:54.926387 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.926389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.926403 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.926464 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:54.926657 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.927674 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.928720 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.929375 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.929447 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.929500 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.929631 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.929670 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260806 22:40:54.930181 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:45345 (local address 127.0.0.1:33764)
0806 22:40:54.925626 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.925664 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.925723 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.926156 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.926158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.926163 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:54.926235 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.926237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.927599 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.927600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.928810 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.928812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.929719 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.929720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.930010 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.930016 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.930071 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.930111 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":3,"thread_start_us":47,"threads_started":1}
I20260806 22:40:54.930349 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43609 (local address 127.0.0.1:54144)
0806 22:40:54.925922 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:54.925930 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:54.925972 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:54.926155 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:54.926158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:54.926160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:54.926235 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.926237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.928545 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.928546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.929306 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:54.929308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.930131 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.930131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:54.930193 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.930197 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:54.930242 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:40:54.930282 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":4,"thread_start_us":50,"threads_started":1}
I20260806 22:40:54.930454 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.0.186.193:45345)
0806 22:40:54.925822 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.925828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.925839 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:54.926042 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.926044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.926059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.926146 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:54.926312 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.927473 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.928890 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.929564 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.929631 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.929683 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.930341 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.930378 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":86,"threads_started":1}
I20260806 22:40:54.930475 17103 tablet_service.cc:1467] Processing CreateTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 (DEFAULT_TABLE table=test-table [id=9babeaa99dd7476780eec0801b14e0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:54.930645 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:54.925771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.0.186.194:43609)
0806 22:40:54.925953 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:54.925956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:54.925966 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:54.926038 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:54.926041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:54.926057 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:54.926146 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:54.927334 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.928423 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.929377 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:54.930067 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:54.930129 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:54.930179 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:54.930524 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 22:40:54.930568 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260806 22:40:54.930855 17103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b03dbc1f9b48f5aa879d258bad6888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:54.931043 16706 tablet_service.cc:1467] Processing CreateTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 (DEFAULT_TABLE table=test-table [id=9babeaa99dd7476780eec0801b14e0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:54.931222 16919 tablet_service.cc:1467] Processing CreateTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 (DEFAULT_TABLE table=test-table [id=9babeaa99dd7476780eec0801b14e0fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:54.931398 16706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b03dbc1f9b48f5aa879d258bad6888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:54.931567 16919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b03dbc1f9b48f5aa879d258bad6888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:54.933988 17215 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Bootstrap starting.
I20260806 22:40:54.934414 17221 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Bootstrap starting.
I20260806 22:40:54.934759 17215 tablet_bootstrap.cc:654] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:54.935048 17221 tablet_bootstrap.cc:654] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:54.935382 17215 log.cc:824] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:54.935400 17217 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Bootstrap starting.
I20260806 22:40:54.935601 17221 log.cc:824] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:54.935999 17217 tablet_bootstrap.cc:654] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:54.936219 17215 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: No bootstrap required, opened a new log
I20260806 22:40:54.936280 17215 ts_tablet_manager.cc:1397] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 22:40:54.936336 17221 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: No bootstrap required, opened a new log
I20260806 22:40:54.936411 17221 ts_tablet_manager.cc:1397] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:54.936558 17217 log.cc:824] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:54.937249 17217 tablet_bootstrap.cc:492] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: No bootstrap required, opened a new log
I20260806 22:40:54.937361 17217 ts_tablet_manager.cc:1397] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:54.938097 17221 raft_consensus.cc:348] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.938265 17221 raft_consensus.cc:374] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:54.938146 17215 raft_consensus.cc:348] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.938289 17221 raft_consensus.cc:729] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6642c5bb2e434fae99a2d9132339a7, State: Initialized, Role: FOLLOWER
I20260806 22:40:54.938309 17215 raft_consensus.cc:374] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:54.938341 17215 raft_consensus.cc:729] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50878e5494aa451bb32a6e373e76dd86, State: Initialized, Role: FOLLOWER
I20260806 22:40:54.938373 17221 consensus_queue.cc:260] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.938446 17215 consensus_queue.cc:260] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.938653 16807 heartbeater.cc:503] Master 127.0.186.254:42703 was elected leader, sending a full tablet report...
I20260806 22:40:54.938725 17215 ts_tablet_manager.cc:1428] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:40:54.938643 17221 ts_tablet_manager.cc:1428] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:54.938839 17010 heartbeater.cc:503] Master 127.0.186.254:42703 was elected leader, sending a full tablet report...
I20260806 22:40:54.939934 17217 raft_consensus.cc:348] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.940065 17217 raft_consensus.cc:374] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:54.940086 17217 raft_consensus.cc:729] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c8b3cbd278475292fb32444f069fe2, State: Initialized, Role: FOLLOWER
I20260806 22:40:54.940182 17217 consensus_queue.cc:260] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:54.940402 17217 ts_tablet_manager.cc:1428] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:54.940446 17194 heartbeater.cc:503] Master 127.0.186.254:42703 was elected leader, sending a full tablet report...
W20260806 22:40:54.994632 17011 tablet.cc:2369] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:55.102802 16809 tablet.cc:2369] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:55.146988 17196 tablet.cc:2369] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:55.172974 17228 raft_consensus.cc:482] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:55.173131 17228 raft_consensus.cc:504] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:55.173504 17228 leader_election.cc:288] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc6642c5bb2e434fae99a2d9132339a7 (127.0.186.193:45345), 50878e5494aa451bb32a6e373e76dd86 (127.0.186.194:43609)
I20260806 22:40:55.176331 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.173793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:50691 (local address 127.0.186.193:45345)
0806 22:40:55.173842 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:55.173844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:55.173853 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:55.173940 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:55.173941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:55.173953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:55.174020 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:55.174185 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.174698 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.175672 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.176034 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.176094 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:55.176103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:55.176232 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:40:55.176265 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:40:55.176331 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.173727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:45345 (local address 127.0.186.195:50691)
0806 22:40:55.173783 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:55.173795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:55.173840 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:55.174050 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:55.174052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:55.174054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:55.174121 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:55.174122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.174772 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.174773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:55.175587 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:55.175589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.176102 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.176103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:55.176173 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:55.176176 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:55.176220 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:40:55.176259 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:40:55.176389 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.173964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:36103 (local address 127.0.186.194:43609)
0806 22:40:55.174004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:55.174006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:55.174065 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:40:55.174100 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:55.174101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:55.174111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:55.174175 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:55.174309 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.174782 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.175725 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.176102 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.176164 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:55.176172 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:55.176288 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:40:55.176326 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:55.176445 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.173849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43609 (local address 127.0.186.195:36103)
0806 22:40:55.174001 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:55.174009 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:55.174055 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:55.174182 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:55.174182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:55.174184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:55.174253 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:55.174254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.174857 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.174858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:55.175649 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:55.175651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.176174 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.176175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:55.176230 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:55.176234 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:55.176277 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:40:55.176313 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":111,"threads_started":1}
I20260806 22:40:55.176709 16741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b03dbc1f9b48f5aa879d258bad6888" candidate_uuid: "43c8b3cbd278475292fb32444f069fe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6642c5bb2e434fae99a2d9132339a7" is_pre_election: true
I20260806 22:40:55.176718 16945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b03dbc1f9b48f5aa879d258bad6888" candidate_uuid: "43c8b3cbd278475292fb32444f069fe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50878e5494aa451bb32a6e373e76dd86" is_pre_election: true
I20260806 22:40:55.176853 16945 raft_consensus.cc:2454] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c8b3cbd278475292fb32444f069fe2 in term 0.
I20260806 22:40:55.176854 16741 raft_consensus.cc:2454] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c8b3cbd278475292fb32444f069fe2 in term 0.
I20260806 22:40:55.177090 17039 leader_election.cc:302] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c8b3cbd278475292fb32444f069fe2, fc6642c5bb2e434fae99a2d9132339a7; no voters:
I20260806 22:40:55.177243 17228 raft_consensus.cc:2781] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:55.177318 17228 raft_consensus.cc:482] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:55.177336 17228 raft_consensus.cc:3037] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:55.178062 17228 raft_consensus.cc:504] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:55.178175 17228 leader_election.cc:288] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [CANDIDATE]: Term 1 election: Requested vote from peers fc6642c5bb2e434fae99a2d9132339a7 (127.0.186.193:45345), 50878e5494aa451bb32a6e373e76dd86 (127.0.186.194:43609)
I20260806 22:40:55.178386 16945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b03dbc1f9b48f5aa879d258bad6888" candidate_uuid: "43c8b3cbd278475292fb32444f069fe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50878e5494aa451bb32a6e373e76dd86"
I20260806 22:40:55.178392 16741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b03dbc1f9b48f5aa879d258bad6888" candidate_uuid: "43c8b3cbd278475292fb32444f069fe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6642c5bb2e434fae99a2d9132339a7"
I20260806 22:40:55.178458 16945 raft_consensus.cc:3037] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:55.178474 16741 raft_consensus.cc:3037] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:55.179152 16945 raft_consensus.cc:2454] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c8b3cbd278475292fb32444f069fe2 in term 1.
I20260806 22:40:55.179210 16741 raft_consensus.cc:2454] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c8b3cbd278475292fb32444f069fe2 in term 1.
I20260806 22:40:55.179364 17039 leader_election.cc:302] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c8b3cbd278475292fb32444f069fe2, fc6642c5bb2e434fae99a2d9132339a7; no voters:
I20260806 22:40:55.179438 17228 raft_consensus.cc:2781] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:55.179594 17228 raft_consensus.cc:686] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 1 LEADER]: Becoming Leader. State: Replica: 43c8b3cbd278475292fb32444f069fe2, State: Running, Role: LEADER
I20260806 22:40:55.179685 17228 consensus_queue.cc:237] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } }
I20260806 22:40:55.180510 16526 catalog_manager.cc:5495] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43c8b3cbd278475292fb32444f069fe2 (127.0.186.195). New cstate: current_term: 1 leader_uuid: "43c8b3cbd278475292fb32444f069fe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43c8b3cbd278475292fb32444f069fe2" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 46195 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:55.213714 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.211450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.0.186.195:46195)
0806 22:40:55.211496 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:55.211498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:55.211507 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:40:55.211576 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:55.211577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:55.211597 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:55.211656 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:55.211786 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.212251 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.212986 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:55.213175 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:55.213229 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:55.213386 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:55.213493 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:55.213565 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:55.213603 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:40:55.213624 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:55.215390 16741 raft_consensus.cc:1261] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 1 FOLLOWER]: Refusing update from remote peer 43c8b3cbd278475292fb32444f069fe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:55.215390 16945 raft_consensus.cc:1261] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 1 FOLLOWER]: Refusing update from remote peer 43c8b3cbd278475292fb32444f069fe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:55.215744 17232 consensus_queue.cc:1035] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50878e5494aa451bb32a6e373e76dd86" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:55.215865 17228 consensus_queue.cc:1035] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6642c5bb2e434fae99a2d9132339a7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:55.218665 17236 mvcc.cc:205] Tried to move back new op lower bound from 7315685602159493120 to 7315685602016612352. Current Snapshot: MvccSnapshot[applied={T|T < 7315685602159493120}]
I20260806 22:40:55.220613 16526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:55.220709 16526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:55.221428 16526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49204da3b22d4b05a9444d86ce367835: Sending DeleteTablet for 3 replicas of tablet f8b03dbc1f9b48f5aa879d258bad6888
I20260806 22:40:55.221788 16706 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:33764
I20260806 22:40:55.222045 17248 tablet_replica.cc:330] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: stopping tablet replica
I20260806 22:40:55.222057 16919 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:54144
I20260806 22:40:55.222157 17248 raft_consensus.cc:2229] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:55.222262 17248 raft_consensus.cc:2258] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:55.222290 17250 tablet_replica.cc:330] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: stopping tablet replica
I20260806 22:40:55.222370 17250 raft_consensus.cc:2229] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:55.222452 17250 raft_consensus.cc:2258] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:55.222457 17103 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b03dbc1f9b48f5aa879d258bad6888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40030
I20260806 22:40:55.222800 17248 ts_tablet_manager.cc:1905] T f8b03dbc1f9b48f5aa879d258bad6888 P fc6642c5bb2e434fae99a2d9132339a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:55.222872 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 16609
I20260806 22:40:55.232342 17253 tablet_replica.cc:330] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: stopping tablet replica
I20260806 22:40:55.232486 17253 raft_consensus.cc:2229] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:55.232616 17253 raft_consensus.cc:2258] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:55.222954 17250 ts_tablet_manager.cc:1905] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:55.233072 17253 ts_tablet_manager.cc:1905] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:55.234248 17250 ts_tablet_manager.cc:1918] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:55.234375 17250 log.cc:1196] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/f8b03dbc1f9b48f5aa879d258bad6888
I20260806 22:40:55.234632 17253 ts_tablet_manager.cc:1918] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:55.234684 17250 ts_tablet_manager.cc:1939] T f8b03dbc1f9b48f5aa879d258bad6888 P 50878e5494aa451bb32a6e373e76dd86: Deleting consensus metadata
I20260806 22:40:55.234799 17253 log.cc:1196] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/f8b03dbc1f9b48f5aa879d258bad6888
I20260806 22:40:55.235132 17253 ts_tablet_manager.cc:1939] T f8b03dbc1f9b48f5aa879d258bad6888 P 43c8b3cbd278475292fb32444f069fe2: Deleting consensus metadata
I20260806 22:40:55.235124 16503 catalog_manager.cc:4841] TS 50878e5494aa451bb32a6e373e76dd86 (127.0.186.194:43609): tablet f8b03dbc1f9b48f5aa879d258bad6888 (table test-table [id=9babeaa99dd7476780eec0801b14e0fc]) successfully deleted
I20260806 22:40:55.235603 16503 catalog_manager.cc:4841] TS 43c8b3cbd278475292fb32444f069fe2 (127.0.186.195:46195): tablet f8b03dbc1f9b48f5aa879d258bad6888 (table test-table [id=9babeaa99dd7476780eec0801b14e0fc]) successfully deleted
W20260806 22:40:55.236937 16503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:45345 (error 108)
I20260806 22:40:55.237147 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 16815
I20260806 22:40:55.237459 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:55.237345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:45345 (local address 127.0.0.1:33780)
0806 22:40:55.237396 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:55.237427 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:45345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:40:55.242341 16503 catalog_manager.cc:4584] TS fc6642c5bb2e434fae99a2d9132339a7 (127.0.186.193:45345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8b03dbc1f9b48f5aa879d258bad6888: Network error: Client connection negotiation failed: client connection to 127.0.186.193:45345: connect: Connection refused (error 111)
I20260806 22:40:55.245360 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 17016
I20260806 22:40:55.257818 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 16473
2026-08-06T22:40:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2230 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:55 dist-test-slave-gjrz krb5kdc[17259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:55 dist-test-slave-gjrz krb5kdc[17259](info): set up 2 sockets
Aug 06 22:40:55 dist-test-slave-gjrz krb5kdc[17259](info): commencing operation
krb5kdc: starting...
W20260806 22:40:56.386276 747 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:40:56 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangeradmin_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangeradmin_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangerlookup_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangerlookup_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
I20260806 22:40:56.540118 747 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwWuo9B/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/postgres -l logfile start
2026-08-06 22:40:58.791 UTC [17297] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:40:58.791 UTC [17297] LOG: listening on IPv4 address "127.0.186.212", port 39523
2026-08-06 22:40:58.795 UTC [17297] LOG: listening on Unix socket "/tmp/.s.PGSQL.39523"
2026-08-06 22:40:58.822 UTC [17302] LOG: database system was shut down at 2026-08-06 22:40:57 UTC
2026-08-06 22:40:58.826 UTC [17297] LOG: database system is ready to accept connections
I20260806 22:40:59.861929 747 mini_postgres.cc:96] Postgres bound to 39523
2026-08-06 22:40:59.868 UTC [17312] FATAL: database "slave" does not exist
127.0.186.212:39523 - accepting connections
I20260806 22:40:59.869390 747 mini_ranger.cc:161] Starting Ranger...
I20260806 22:40:59.884210 747 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:41:00.327397 747 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:41:00.327469 747 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin
2026-08-06 22:41:00,610 [I] DB FLAVOR :POSTGRES
2026-08-06 22:41:00,611 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:41:00,611 [I] Checking connection..
2026-08-06 22:41:00,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:41:00,868 [I] Checking connection passed.
2026-08-06 22:41:00,868 [I] --------- Verifying version history table ---------
2026-08-06 22:41:00,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:01,128 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:41:01,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:01,370 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:41:01,370 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:41:01,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:41:01.624 UTC [17397] WARNING: there is no transaction in progress
2026-08-06 22:41:01,630 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:41:01,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:01,889 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:41:01,890 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:41:01,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:41:02,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:41:02,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-06 22:41:02,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:41:02,905 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:41:02,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:41:03,149 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:41:03,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:41:03,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:41:03,630 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:41:03,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:41:04.571 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.586 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.603 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.617 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.633 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.650 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.704 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.711 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.723 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.733 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.742 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.754 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.764 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.770 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.777 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:04.783 UTC [17554] WARNING: there is no transaction in progress
2026-08-06 22:41:05,394 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:41:05,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:41:05,646 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:41:05,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:41:05,894 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:41:05,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:41:06,125 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:41:06,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:41:06,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:41:06,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-06 22:41:06,842 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:41:06,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:41:07,096 [I] DB_PATCHES have already been applied
I20260806 22:41:07.101716 747 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwWuo9B/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:41:07.101783 747 mini_ranger.cc:193] Using host: 127.0.186.212
I20260806 22:41:07.105536 747 mini_ranger.cc:227] Ranger admin URL: http://127.0.186.212:50849
Aug 06, 2026 10:41:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:41:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:41:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:41:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:41:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:41:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:41:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:41:08.373438 747 mini_ranger.cc:160] Time spent starting Ranger: real 8.504s user 0.002s sys 0.008s
Aug 06, 2026 10:41:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:41:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 10:41:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:41:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:41:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:41:29.925309 747 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangerkms_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/rangerkms_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.186.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.186.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:41:29 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056089, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:41:30.003696 747 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:41:30.013816 747 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:41:30.751317 747 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:41:30.751407 747 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms
I20260806 22:41:30.751425 747 mini_ranger_kms.cc:227] Using postgres at 127.0.186.212:39523
2026-08-06 22:41:31,329 [I] DB FLAVOR :POSTGRES
2026-08-06 22:41:31,329 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:41:31,329 [I] Checking connection
2026-08-06 22:41:31,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:41:31,645 [I] connection success
2026-08-06 22:41:31,646 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:41:31,646 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:41:31,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:41:31,886 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:41:31,887 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:41:31,887 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:41:31,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:39523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:41:32,150 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:41:32.626463 747 mini_ranger_kms.cc:310] Created kms service
I20260806 22:41:32.799194 747 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:41:32.841452 747 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:41:32 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056089, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:41:33.244351 747 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:41:33.244472 747 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:41:33.244494 747 mini_ranger_kms.cc:240] Using host: 127.0.186.212
I20260806 22:41:33.249096 747 mini_ranger_kms.cc:276] Ranger KMS PID: 17828
I20260806 22:41:33.249272 747 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.186.212:56213
Aug 06, 2026 10:41:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:41:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:41:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:41:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:41:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:41:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:41:35.765787 747 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.762s user 0.018s sys 0.530s
I20260806 22:41:35.765900 747 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:41:35.765985 747 mini_ranger_kms.cc:401] 127.0.186.212:56213/kms/v1/keys
Aug 06 22:41:35 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:41:36 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
Aug 06 22:41:36 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
Aug 06 22:41:39 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:41:39.147876 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42655
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42655
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56213/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:39.274425 17900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:39.274654 17900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:39.274689 17900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:39.276021 17900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:39.276067 17900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:39.276087 17900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:39.276103 17900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:39.276117 17900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:39.278053 17900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.186.212:56213/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42655
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:41:39.278303 17900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:39.278514 17900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:39.283349 17907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:39.283473 17908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:39.283537 17910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:39.284000 17900 server_base.cc:1034] running on GCE node
I20260806 22:41:39.284281 17900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:39.284632 17900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:39.285796 17900 hybrid_clock.cc:648] HybridClock initialized: now 1786056099285783 us; error 28 us; skew 500 ppm
Aug 06 22:41:39 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:39.293358 17900 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:41:39.293912 17900 webserver.cc:460] Webserver started at http://127.0.186.254:38253/ using document root <none> and password file <none>
I20260806 22:41:39.294164 17900 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:39.294217 17900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:39.294343 17900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:41:39 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:41:39.346390 17900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "edbc29ab933c494fb30fb9290f52035d"
format_stamp: "Formatted at 2026-08-06 22:41:39 on dist-test-slave-gjrz"
server_key: "61657cd59aeff70d43859a8291775e68"
server_key_iv: "363be2be19d07b0610d65af83a5fa230"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:39.346830 17900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "edbc29ab933c494fb30fb9290f52035d"
format_stamp: "Formatted at 2026-08-06 22:41:39 on dist-test-slave-gjrz"
server_key: "61657cd59aeff70d43859a8291775e68"
server_key_iv: "363be2be19d07b0610d65af83a5fa230"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:39.348615 17900 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 22:41:39.365500 17923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:39.365727 17900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:41:39.365835 17900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "edbc29ab933c494fb30fb9290f52035d"
format_stamp: "Formatted at 2026-08-06 22:41:39 on dist-test-slave-gjrz"
server_key: "61657cd59aeff70d43859a8291775e68"
server_key_iv: "363be2be19d07b0610d65af83a5fa230"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:39.365944 17900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:39.386509 17900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:39.388626 17900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:39.388845 17900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:39.401423 17900 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42655
I20260806 22:41:39.401558 17998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42655 every 8 connection(s)
I20260806 22:41:39.401975 17900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:39.403035 17999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:39.405359 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 17900
I20260806 22:41:39.405481 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:41:39.405687 747 external_mini_cluster.cc:1428] Setting key 4b4f56ffb0c5dd2769afb0a8bb5d7442
I20260806 22:41:39.406781 17999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: Bootstrap starting.
I20260806 22:41:39.408305 17999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:39.409173 17999 log.cc:824] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:39.410104 17999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: No bootstrap required, opened a new log
I20260806 22:41:39.414096 17999 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } }
I20260806 22:41:39.414237 17999 raft_consensus.cc:374] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:39.414271 17999 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbc29ab933c494fb30fb9290f52035d, State: Initialized, Role: FOLLOWER
I20260806 22:41:39.414443 17999 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } }
I20260806 22:41:39.414513 17999 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:39.414546 17999 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:39.414605 17999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:41:39 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:41:39.415403 17999 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } }
I20260806 22:41:39.415664 17999 leader_election.cc:302] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbc29ab933c494fb30fb9290f52035d; no voters:
I20260806 22:41:39.415853 17999 leader_election.cc:288] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:39.415997 18010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:39.416440 17999 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:39.416471 18010 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [term 1 LEADER]: Becoming Leader. State: Replica: edbc29ab933c494fb30fb9290f52035d, State: Running, Role: LEADER
I20260806 22:41:39.416704 18010 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } }
I20260806 22:41:39.417183 18010 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edbc29ab933c494fb30fb9290f52035d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } } }
I20260806 22:41:39.417289 18013 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbc29ab933c494fb30fb9290f52035d. Latest consensus state: current_term: 1 leader_uuid: "edbc29ab933c494fb30fb9290f52035d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbc29ab933c494fb30fb9290f52035d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42655 } } }
I20260806 22:41:39.417447 18013 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:39.417330 18010 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:39.417865 18017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:39.419039 18017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:39.421378 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:39.406371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.0.186.254:42655)
0806 22:41:39.406693 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:39.406702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:39.406734 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:41:39.407815 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:39.407825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:39.407894 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:39.408051 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:39.408942 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:39.410496 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:39.411602 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:39.412445 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:39.412558 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:39.415515 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:39.415519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:39.415519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:39.415531 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:39.418263 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:39.419254 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:39.419257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:39.419258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:39.419422 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:39.419726 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:39.419727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:39.419727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:39.420057 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:39.420998 (+ 941us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:39.421066 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:41:39.421156 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":132,"threads_started":1}
I20260806 22:41:39.422693 18017 catalog_manager.cc:1347] Generated new cluster ID: 0fa24c036e3344fdaf92327e58938bc6
I20260806 22:41:39.422734 18017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:39.423579 17999 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:41:39.434844 18017 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:39.435985 18017 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:39.446720 18017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: Generated new TSK 0
I20260806 22:41:39.446954 18017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:41:40.442296 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56213/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:40.585475 18066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:40.585704 18066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:40.585736 18066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:40.586930 18066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:40.586970 18066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:40.587006 18066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:41:40.588778 18066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.186.212:56213/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:41:40.589031 18066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:40.589305 18066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:40.597136 18076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.597096 18072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.597139 18071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:40.597409 18066 server_base.cc:1034] running on GCE node
I20260806 22:41:40.597599 18066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:40.597815 18066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:40.598965 18066 hybrid_clock.cc:648] HybridClock initialized: now 1786056100598955 us; error 29 us; skew 500 ppm
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:40.602145 18066 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:41:40.602595 18066 webserver.cc:460] Webserver started at http://127.0.186.193:36769/ using document root <none> and password file <none>
I20260806 22:41:40.602823 18066 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:40.602866 18066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:40.602979 18066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:41:40.619481 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "95ae4a80ff0e4e2ca69de0be97e910c1"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "b4f6b8fafba4d8428e57e3650658b5a6"
server_key_iv: "e1f3d55b8703afdfd20a714c5f2ac9f6"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.619971 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "95ae4a80ff0e4e2ca69de0be97e910c1"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "b4f6b8fafba4d8428e57e3650658b5a6"
server_key_iv: "e1f3d55b8703afdfd20a714c5f2ac9f6"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.621891 18066 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 22:41:40.639451 18093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:40.639616 18066 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 22:41:40.639708 18066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "95ae4a80ff0e4e2ca69de0be97e910c1"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "b4f6b8fafba4d8428e57e3650658b5a6"
server_key_iv: "e1f3d55b8703afdfd20a714c5f2ac9f6"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.639793 18066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:40.668723 18066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:40.670790 18066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:40.671007 18066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:40.671478 18066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:40.672013 18066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:40.672114 18066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:40.672217 18066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:40.672370 18066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:40.688681 18066 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:44599
I20260806 22:41:40.689386 18066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:40.690425 18269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:44599 every 8 connection(s)
I20260806 22:41:40.694816 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 18066
I20260806 22:41:40.694952 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:40.695168 747 external_mini_cluster.cc:1428] Setting key 9edc92d0d18ef268a47dc94f2c729f8c
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:41:40.703758 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:40.691416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42655 (local address 127.0.186.193:44079)
0806 22:41:40.691798 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:40.691856 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:40.692641 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:40.693106 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:40.693111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:40.693495 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:40.693975 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:40.693983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.694972 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.694974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:40.696312 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:40.696315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.696931 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.696932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:40.697111 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:40.697842 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:40.697853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:40.699748 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:40.701962 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:40.701969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:40.701971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:40.702283 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:40.702548 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:40.702549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:40.702549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:40.702695 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:40.703002 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:40.703009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:40.703180 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 22:41:40.703316 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 22:41:40.703507 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":163,"threads_started":1}
I20260806 22:41:40.704507 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:40.691416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:44079 (local address 127.0.186.254:42655)
0806 22:41:40.692843 (+ 1427us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:40.692845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:40.692853 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:41:40.692879 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:40.692880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:40.692892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:40.692969 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:40.694096 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.694853 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.696417 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.696774 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.696853 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:40.699860 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:40.699862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:40.699863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:40.699872 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:40.701832 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.702394 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.702394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.702395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.702466 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.702776 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.702777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.702777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.702911 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:40.704258 (+ 1347us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:40.704304 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:41:40.704406 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":63,"threads_started":1}
I20260806 22:41:40.705142 18272 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42655
I20260806 22:41:40.705243 18272 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:40.705431 18272 heartbeater.cc:511] Master 127.0.186.254:42655 requested a full tablet report, sending...
I20260806 22:41:40.706029 17947 ts_manager.cc:194] Registered new tserver with Master: 95ae4a80ff0e4e2ca69de0be97e910c1 (127.0.186.193:44599)
I20260806 22:41:40.706926 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:44079
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:41:40.750038 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56213/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:41:40.933269 18281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:40.933663 18281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:40.933758 18281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:40.935313 18281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:40.935362 18281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:40.935401 18281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:41:40.937615 18281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.186.212:56213/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:41:40.938052 18281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:40.938393 18281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:40.949270 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.949563 18290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.950070 18288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:40.952587 18281 server_base.cc:1034] running on GCE node
I20260806 22:41:40.952843 18281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:40.953162 18281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:40.954385 18281 hybrid_clock.cc:648] HybridClock initialized: now 1786056100954362 us; error 33 us; skew 500 ppm
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:40.958108 18281 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:41:40.958627 18281 webserver.cc:460] Webserver started at http://127.0.186.194:45277/ using document root <none> and password file <none>
I20260806 22:41:40.958860 18281 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:40.958912 18281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:40.959043 18281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:41:40 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:41:40.973490 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "1a89d336d6a844ce912f3b0ebf5f1c63"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "234071d618a7e0ad4b9c063d95d1d083"
server_key_iv: "0613f0cfc342b7d9251dad28f62c2400"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.973937 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "1a89d336d6a844ce912f3b0ebf5f1c63"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "234071d618a7e0ad4b9c063d95d1d083"
server_key_iv: "0613f0cfc342b7d9251dad28f62c2400"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.978446 18281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:41:40.993805 18304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:40.994056 18281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 22:41:40.994148 18281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "1a89d336d6a844ce912f3b0ebf5f1c63"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-gjrz"
server_key: "234071d618a7e0ad4b9c063d95d1d083"
server_key_iv: "0613f0cfc342b7d9251dad28f62c2400"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:40.994236 18281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:41.019554 18281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:41.022159 18281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:41.022374 18281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:41.022737 18281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:41.023253 18281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:41.023370 18281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.023439 18281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:41.023494 18281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.036963 18281 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:33867
I20260806 22:41:41.037351 18466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:33867 every 8 connection(s)
I20260806 22:41:41.037508 18281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:41.044366 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 18281
I20260806 22:41:41.044507 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:41.044704 747 external_mini_cluster.cc:1428] Setting key 096a5bfc328dca8761b62c17bffbfaa9
Aug 06 22:41:41 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:41:41.054248 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.038653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:53653 (local address 127.0.186.254:42655)
0806 22:41:41.038706 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.038708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.039136 (+ 428us) server_negotiation.cc:407] Connection header received
0806 22:41:41.040569 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.040571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.040591 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.040624 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:41.041780 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.042588 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.044241 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.044569 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.044622 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.050768 (+ 6146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:41.050770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:41.050770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:41.050780 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:41.052554 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.053079 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.053080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.053081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.053148 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.053450 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.053451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.053451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.053598 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:41.053654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.054049 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.054150 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:41:41.055050 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.038632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42655 (local address 127.0.186.194:53653)
0806 22:41:41.038928 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.038985 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.040406 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.040832 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.040837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.041236 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:41.041649 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.041658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.042715 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.042716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.044122 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.044126 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.044774 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.044775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.047990 (+ 3215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.048714 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:41.048725 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:41.050635 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:41.052653 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:41.052661 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:41.052664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:41.052980 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:41.053220 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:41.053221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:41.053221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:41.053364 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:41.053691 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:41.053698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:41.053867 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.054591 (+ 724us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.054770 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":124,"threads_started":1}
I20260806 22:41:41.055791 18467 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42655
I20260806 22:41:41.055902 18467 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:41.056133 18467 heartbeater.cc:511] Master 127.0.186.254:42655 requested a full tablet report, sending...
I20260806 22:41:41.056588 17947 ts_manager.cc:194] Registered new tserver with Master: 1a89d336d6a844ce912f3b0ebf5f1c63 (127.0.186.194:33867)
I20260806 22:41:41.057085 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:53653
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:41:41.088694 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56213/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:41.247823 18480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:41.248204 18480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:41.248312 18480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:41.250074 18480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:41.250191 18480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:41.250285 18480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:41:41.252619 18480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.186.212:56213/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:41:41.253142 18480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:41.253508 18480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:41.261860 18488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:41.262427 18487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:41.265902 18493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:41.267004 18480 server_base.cc:1034] running on GCE node
I20260806 22:41:41.267199 18480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:41.267478 18480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:41.271062 18480 hybrid_clock.cc:648] HybridClock initialized: now 1786056101271029 us; error 45 us; skew 500 ppm
Aug 06 22:41:41 dist-test-slave-gjrz krb5kdc[17259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:41.275112 18480 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:41:41.275735 18480 webserver.cc:460] Webserver started at http://127.0.186.195:36017/ using document root <none> and password file <none>
I20260806 22:41:41.276048 18480 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:41.276151 18480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:41.276325 18480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:41:41 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:41:41.301247 18480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "12c1e3e89a764bb482c4f5f411ab2865"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-gjrz"
server_key: "48a60cd173c552e38dac4ad4ad3116dc"
server_key_iv: "077ab699e57c7746c0ec41cae57029b5"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:41.301895 18480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "12c1e3e89a764bb482c4f5f411ab2865"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-gjrz"
server_key: "48a60cd173c552e38dac4ad4ad3116dc"
server_key_iv: "077ab699e57c7746c0ec41cae57029b5"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:41.304231 18480 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:41.318403 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.318632 18480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:41.318732 18480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "12c1e3e89a764bb482c4f5f411ab2865"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-gjrz"
server_key: "48a60cd173c552e38dac4ad4ad3116dc"
server_key_iv: "077ab699e57c7746c0ec41cae57029b5"
server_key_version: "kuduclusterkey@0"
I20260806 22:41:41.318826 18480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:41.336675 18480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:41.339116 18480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:41.339314 18480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:41.339638 18480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:41.340142 18480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:41.340195 18480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.340271 18480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:41.340303 18480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.354465 18480 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45659
I20260806 22:41:41.355106 18480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:41.356611 18688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45659 every 8 connection(s)
I20260806 22:41:41.358727 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 18480
I20260806 22:41:41.358845 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:41.359022 747 external_mini_cluster.cc:1428] Setting key 628c26fb59ef78c9a78660fe871b3cf6
Aug 06 22:41:41 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:41:41.365638 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.356582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:59619 (local address 127.0.186.254:42655)
0806 22:41:41.356630 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.356632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.356998 (+ 366us) server_negotiation.cc:407] Connection header received
0806 22:41:41.357615 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.357616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.357629 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.357685 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:41.358538 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.359329 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.360609 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.360814 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.360868 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.363194 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:41.363195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:41.363196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:41.363203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:41.364446 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.364797 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.364798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.364798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.364855 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.365066 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.365067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.365068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.365170 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:41.365224 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.365473 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.365551 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:41:41.365878 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.356176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42655 (local address 127.0.186.195:59619)
0806 22:41:41.356859 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.356894 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.357510 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.357811 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.357815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.358096 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:41.358379 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.358384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.359433 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.359434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.360526 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.360529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.360884 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.360885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.361074 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.361618 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:41.361626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:41.363113 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:41.364519 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:41.364525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:41.364527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:41.364732 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:41.364908 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:41.364909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:41.364910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:41.365011 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:41.365222 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:41.365229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:41.365360 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.365453 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.365592 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":182,"threads_started":1}
I20260806 22:41:41.366520 18689 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42655
I20260806 22:41:41.366613 18689 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:41.366786 18689 heartbeater.cc:511] Master 127.0.186.254:42655 requested a full tablet report, sending...
I20260806 22:41:41.367156 17947 ts_manager.cc:194] Registered new tserver with Master: 12c1e3e89a764bb482c4f5f411ab2865 (127.0.186.195:45659)
I20260806 22:41:41.367539 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:59619
I20260806 22:41:41.370616 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:41.378121 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.372015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.0.186.254:42655)
0806 22:41:41.372060 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.372062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.372216 (+ 154us) server_negotiation.cc:407] Connection header received
0806 22:41:41.372273 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.372274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.372284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.372327 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:41.372885 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.373335 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.374363 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.374574 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.374611 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.375989 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:41.375990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:41.375990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:41.375997 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:41.377184 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.377479 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.377480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.377481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.377530 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:41.377688 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:41.377689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:41.377689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:41.377787 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:41.377866 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.377968 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.378037 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:41:41.379647 17947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:41.381363 17947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:41.390858 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.384006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:33867 (local address 127.0.0.1:41720)
0806 22:41:41.385194 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.385243 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.385316 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.385942 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.385955 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.385958 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:41.386023 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.386026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.387604 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.387605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.388775 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.388778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.390227 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.390228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.390306 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.390670 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.390732 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.390780 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"mutex_wait_us":335,"thread_start_us":103,"threads_started":1}
I20260806 22:41:41.391676 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.384629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.0.186.194:33867)
0806 22:41:41.385068 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.385073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.385438 (+ 365us) server_negotiation.cc:407] Connection header received
0806 22:41:41.385525 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.385528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.385545 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.385620 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:41.386112 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.387526 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.388900 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.389712 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.389780 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.389891 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.391530 (+ 1639us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.391574 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":152,"threads_started":1}
I20260806 22:41:41.392539 18373 tablet_service.cc:1467] Processing CreateTablet for tablet c31e7802e5aa48cca912ec111430b5ef (DEFAULT_TABLE table=test-table [id=cb9ce5b523f74c15bd72bd0521c22a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:41.392962 18373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31e7802e5aa48cca912ec111430b5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:41.396296 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.384008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45659 (local address 127.0.0.1:54454)
0806 22:41:41.387394 (+ 3386us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.387404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.387460 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.388858 (+ 1398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.388859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.388861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:41.388898 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.388899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.390205 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.390206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.391421 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.391424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.396076 (+ 4652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.396077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.396130 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.396134 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.396189 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.396228 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3347,"thread_start_us":106,"threads_started":1}
I20260806 22:41:41.396956 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.384406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.0.186.195:45659)
0806 22:41:41.385881 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.385898 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.387476 (+ 1578us) server_negotiation.cc:407] Connection header received
0806 22:41:41.387557 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.387560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.387754 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.387831 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:41.388978 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.390142 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.392032 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.393124 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.393452 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.393560 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.396819 (+ 3259us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.396861 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":116,"threads_started":1}
I20260806 22:41:41.397895 18590 tablet_service.cc:1467] Processing CreateTablet for tablet c31e7802e5aa48cca912ec111430b5ef (DEFAULT_TABLE table=test-table [id=cb9ce5b523f74c15bd72bd0521c22a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:41.398375 18590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31e7802e5aa48cca912ec111430b5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:41.407491 18714 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Bootstrap starting.
I20260806 22:41:41.408875 18714 tablet_bootstrap.cc:654] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:41.409330 18714 log.cc:824] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:41.410904 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.386115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44599 (local address 127.0.0.1:32972)
0806 22:41:41.390419 (+ 4304us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.390429 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.390473 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.391497 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.391498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.391500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:41.391534 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.391535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.394886 (+ 3351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.394887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.395986 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.395990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.410676 (+ 14686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.410678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.410732 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.410737 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.410786 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.410835 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4259,"thread_start_us":106,"threads_started":1}
I20260806 22:41:41.410923 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.386406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.0.186.193:44599)
0806 22:41:41.391327 (+ 4921us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.391332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.391341 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:41:41.391374 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.391376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.391391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.393596 (+ 2205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:41.393669 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.394788 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.397262 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.410455 (+ 13193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.410575 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.410647 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.410818 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.410860 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4851,"thread_start_us":72,"threads_started":1}
I20260806 22:41:41.411660 18714 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: No bootstrap required, opened a new log
I20260806 22:41:41.411732 18714 ts_tablet_manager.cc:1397] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 22:41:41.411834 18173 tablet_service.cc:1467] Processing CreateTablet for tablet c31e7802e5aa48cca912ec111430b5ef (DEFAULT_TABLE table=test-table [id=cb9ce5b523f74c15bd72bd0521c22a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:41.412211 18173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31e7802e5aa48cca912ec111430b5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:41.415138 18714 raft_consensus.cc:348] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.415658 18714 raft_consensus.cc:374] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:41.415735 18714 raft_consensus.cc:729] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a89d336d6a844ce912f3b0ebf5f1c63, State: Initialized, Role: FOLLOWER
I20260806 22:41:41.415885 18714 consensus_queue.cc:260] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.416190 18714 ts_tablet_manager.cc:1428] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 22:41:41.416312 18467 heartbeater.cc:503] Master 127.0.186.254:42655 was elected leader, sending a full tablet report...
I20260806 22:41:41.416715 18719 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Bootstrap starting.
I20260806 22:41:41.417936 18719 tablet_bootstrap.cc:654] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:41.418445 18719 log.cc:824] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:41.419297 18719 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: No bootstrap required, opened a new log
I20260806 22:41:41.419368 18719 ts_tablet_manager.cc:1397] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:41:41.422264 18719 raft_consensus.cc:348] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.422397 18719 raft_consensus.cc:374] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:41.422436 18719 raft_consensus.cc:729] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ae4a80ff0e4e2ca69de0be97e910c1, State: Initialized, Role: FOLLOWER
I20260806 22:41:41.422570 18719 consensus_queue.cc:260] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.422821 18719 ts_tablet_manager.cc:1428] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:41.422855 18272 heartbeater.cc:503] Master 127.0.186.254:42655 was elected leader, sending a full tablet report...
I20260806 22:41:41.435465 18715 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Bootstrap starting.
I20260806 22:41:41.437175 18715 tablet_bootstrap.cc:654] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:41.438109 18715 log.cc:824] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Log is configured to *not* fsync() on all Append() calls
W20260806 22:41:41.441294 18273 tablet.cc:2369] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:41.447640 18715 tablet_bootstrap.cc:492] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: No bootstrap required, opened a new log
I20260806 22:41:41.447733 18715 ts_tablet_manager.cc:1397] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20260806 22:41:41.451241 18715 raft_consensus.cc:348] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.454279 18715 raft_consensus.cc:374] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:41.454391 18715 raft_consensus.cc:729] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c1e3e89a764bb482c4f5f411ab2865, State: Initialized, Role: FOLLOWER
I20260806 22:41:41.454624 18715 consensus_queue.cc:260] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.455077 18715 ts_tablet_manager.cc:1428] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20260806 22:41:41.455252 18689 heartbeater.cc:503] Master 127.0.186.254:42655 was elected leader, sending a full tablet report...
W20260806 22:41:41.549019 18469 tablet.cc:2369] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:41:41 dist-test-slave-gjrz krb5kdc[17259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
W20260806 22:41:41.607077 18690 tablet.cc:2369] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:41.713963 18717 raft_consensus.cc:482] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:41.714116 18717 raft_consensus.cc:504] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.714547 18717 leader_election.cc:288] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12c1e3e89a764bb482c4f5f411ab2865 (127.0.186.195:45659), 95ae4a80ff0e4e2ca69de0be97e910c1 (127.0.186.193:44599)
I20260806 22:41:41.719408 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.714939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:60543 (local address 127.0.186.193:44599)
0806 22:41:41.714989 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.714991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.715086 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:41:41.715184 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.715185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.715194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.715219 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:41.715702 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.716498 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.718096 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.718817 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.718891 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.718901 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.719235 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.719318 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:41:41.719972 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.714760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44599 (local address 127.0.186.194:60543)
0806 22:41:41.715016 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.715027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.715116 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.715301 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.715302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.715304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:41.715591 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.715596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.716626 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.716627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.717999 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.718002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.718983 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.718984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.719131 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.719136 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.719795 (+ 659us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.719842 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":63,"threads_started":1}
I20260806 22:41:41.720340 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.714979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:40225 (local address 127.0.186.195:45659)
0806 22:41:41.715018 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.715020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.715132 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:41:41.715245 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.715246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.715259 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.715314 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:41.715580 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.716353 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.718049 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.719950 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.720000 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.720009 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.720232 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.720272 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:41.720520 18178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31e7802e5aa48cca912ec111430b5ef" candidate_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" is_pre_election: true
I20260806 22:41:41.720696 18178 raft_consensus.cc:2454] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a89d336d6a844ce912f3b0ebf5f1c63 in term 0.
I20260806 22:41:41.720765 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.714760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45659 (local address 127.0.186.194:40225)
0806 22:41:41.715064 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.715075 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:41.715162 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:41.715410 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:41.715411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:41.715413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:41.715492 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.715494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.716463 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.716464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.717919 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:41.717922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.720071 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.720072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:41.720148 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.720151 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:41.720616 (+ 465us) client_negotiation.cc:240] Negotiation successful
0806 22:41:41.720665 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":124,"thread_start_us":80,"threads_started":1}
I20260806 22:41:41.721016 18306 leader_election.cc:302] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a89d336d6a844ce912f3b0ebf5f1c63, 95ae4a80ff0e4e2ca69de0be97e910c1; no voters:
I20260806 22:41:41.721065 18622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31e7802e5aa48cca912ec111430b5ef" candidate_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c1e3e89a764bb482c4f5f411ab2865" is_pre_election: true
I20260806 22:41:41.721167 18717 raft_consensus.cc:2781] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:41.721213 18717 raft_consensus.cc:482] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:41.721210 18622 raft_consensus.cc:2454] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a89d336d6a844ce912f3b0ebf5f1c63 in term 0.
I20260806 22:41:41.721237 18717 raft_consensus.cc:3037] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:41.722721 18717 raft_consensus.cc:504] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.722863 18717 leader_election.cc:288] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [CANDIDATE]: Term 1 election: Requested vote from peers 12c1e3e89a764bb482c4f5f411ab2865 (127.0.186.195:45659), 95ae4a80ff0e4e2ca69de0be97e910c1 (127.0.186.193:44599)
I20260806 22:41:41.723284 18622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31e7802e5aa48cca912ec111430b5ef" candidate_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12c1e3e89a764bb482c4f5f411ab2865"
I20260806 22:41:41.723377 18622 raft_consensus.cc:3037] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:41.723450 18178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31e7802e5aa48cca912ec111430b5ef" candidate_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1"
I20260806 22:41:41.723527 18178 raft_consensus.cc:3037] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:41.724669 18178 raft_consensus.cc:2454] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a89d336d6a844ce912f3b0ebf5f1c63 in term 1.
I20260806 22:41:41.724684 18622 raft_consensus.cc:2454] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a89d336d6a844ce912f3b0ebf5f1c63 in term 1.
I20260806 22:41:41.724859 18306 leader_election.cc:302] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a89d336d6a844ce912f3b0ebf5f1c63, 95ae4a80ff0e4e2ca69de0be97e910c1; no voters:
I20260806 22:41:41.725009 18717 raft_consensus.cc:2781] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:41.725171 18717 raft_consensus.cc:686] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 1 LEADER]: Becoming Leader. State: Replica: 1a89d336d6a844ce912f3b0ebf5f1c63, State: Running, Role: LEADER
I20260806 22:41:41.725276 18717 consensus_queue.cc:237] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } }
I20260806 22:41:41.726145 17947 catalog_manager.cc:5495] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a89d336d6a844ce912f3b0ebf5f1c63 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a89d336d6a844ce912f3b0ebf5f1c63" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 33867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 } health_report { overall_health: UNKNOWN } } }
I20260806 22:41:41.745121 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.741225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.0.186.194:33867)
0806 22:41:41.741291 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.741293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.741339 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:41:41.741379 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.741380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.741402 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.741475 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:41.741671 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.742429 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.743849 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.744181 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.744255 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.744477 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:41.744614 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:41.744683 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.744929 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.744953 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:41:41.747898 18178 raft_consensus.cc:1261] T c31e7802e5aa48cca912ec111430b5ef P 95ae4a80ff0e4e2ca69de0be97e910c1 [term 1 FOLLOWER]: Refusing update from remote peer 1a89d336d6a844ce912f3b0ebf5f1c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:41.748059 18622 raft_consensus.cc:1261] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 1 FOLLOWER]: Refusing update from remote peer 1a89d336d6a844ce912f3b0ebf5f1c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:41.748296 18717 consensus_queue.cc:1035] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ae4a80ff0e4e2ca69de0be97e910c1" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:41.748509 18732 consensus_queue.cc:1035] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12c1e3e89a764bb482c4f5f411ab2865" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:41.752089 18741 mvcc.cc:205] Tried to move back new op lower bound from 7315685792755879936 to 7315685792667340800. Current Snapshot: MvccSnapshot[applied={T|T < 7315685792755879936}]
I20260806 22:41:41.754730 18738 mvcc.cc:205] Tried to move back new op lower bound from 7315685792755879936 to 7315685792667340800. Current Snapshot: MvccSnapshot[applied={T|T < 7315685792755879936}]
I20260806 22:41:41.757763 17947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:41.757872 17947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:41.758656 17947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbc29ab933c494fb30fb9290f52035d: Sending DeleteTablet for 3 replicas of tablet c31e7802e5aa48cca912ec111430b5ef
I20260806 22:41:41.759066 18590 tablet_service.cc:1514] Processing DeleteTablet for tablet c31e7802e5aa48cca912ec111430b5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:41 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:54454
I20260806 22:41:41.759245 18373 tablet_service.cc:1514] Processing DeleteTablet for tablet c31e7802e5aa48cca912ec111430b5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:41 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:41720
I20260806 22:41:41.759478 18173 tablet_service.cc:1514] Processing DeleteTablet for tablet c31e7802e5aa48cca912ec111430b5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:41 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:32972
I20260806 22:41:41.760192 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 18066
I20260806 22:41:41.767405 18754 tablet_replica.cc:330] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: stopping tablet replica
I20260806 22:41:41.767529 18754 raft_consensus.cc:2229] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:41:41.767665 18754 raft_consensus.cc:2258] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:41.767948 18752 tablet_replica.cc:330] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: stopping tablet replica
I20260806 22:41:41.768044 18752 raft_consensus.cc:2229] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:41.768128 18752 raft_consensus.cc:2258] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:41.768414 18754 ts_tablet_manager.cc:1905] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:41.770567 18754 ts_tablet_manager.cc:1918] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:41.770642 18754 log.cc:1196] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/c31e7802e5aa48cca912ec111430b5ef
I20260806 22:41:41.770953 18754 ts_tablet_manager.cc:1939] T c31e7802e5aa48cca912ec111430b5ef P 1a89d336d6a844ce912f3b0ebf5f1c63: Deleting consensus metadata
I20260806 22:41:41.771807 18752 ts_tablet_manager.cc:1905] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:41.774159 18752 ts_tablet_manager.cc:1918] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:41.774228 18752 log.cc:1196] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/c31e7802e5aa48cca912ec111430b5ef
I20260806 22:41:41.774243 17926 catalog_manager.cc:4841] TS 1a89d336d6a844ce912f3b0ebf5f1c63 (127.0.186.194:33867): tablet c31e7802e5aa48cca912ec111430b5ef (table test-table [id=cb9ce5b523f74c15bd72bd0521c22a0c]) successfully deleted
I20260806 22:41:41.774485 18752 ts_tablet_manager.cc:1939] T c31e7802e5aa48cca912ec111430b5ef P 12c1e3e89a764bb482c4f5f411ab2865: Deleting consensus metadata
W20260806 22:41:41.774760 17924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:44599 (error 108)
I20260806 22:41:41.775004 17928 catalog_manager.cc:4841] TS 12c1e3e89a764bb482c4f5f411ab2865 (127.0.186.195:45659): tablet c31e7802e5aa48cca912ec111430b5ef (table test-table [id=cb9ce5b523f74c15bd72bd0521c22a0c]) successfully deleted
I20260806 22:41:41.775550 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 18281
I20260806 22:41:41.786180 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 18480
I20260806 22:41:41.788687 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.786348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44599 (local address 127.0.0.1:32988)
0806 22:41:41.788613 (+ 2265us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:41.788654 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2246}
W20260806 22:41:41.789665 17924 catalog_manager.cc:4584] TS 95ae4a80ff0e4e2ca69de0be97e910c1 (127.0.186.193:44599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c31e7802e5aa48cca912ec111430b5ef: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44599: connect: Connection refused (error 111)
I20260806 22:41:41.798307 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 17900
I20260806 22:41:41.807219 747 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:41:42.151145 747 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:41:42.151247 747 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:41:42.495136 747 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:41:42.495 UTC [17297] LOG: received smart shutdown request
2026-08-06 22:41:42.498 UTC [17297] LOG: background worker "logical replication launcher" (PID 17309) exited with exit code 1
2026-08-06 22:41:42.499 UTC [17303] LOG: shutting down
2026-08-06 22:41:42.655 UTC [17297] LOG: database system is shut down
2026-08-06T22:41:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47481 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:42 dist-test-slave-gjrz krb5kdc[18769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:42 dist-test-slave-gjrz krb5kdc[18769](info): set up 2 sockets
Aug 06 22:41:42 dist-test-slave-gjrz krb5kdc[18769](info): commencing operation
krb5kdc: starting...
W20260806 22:41:43.888396 747 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:41:43 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangeradmin_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangeradmin_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangerlookup_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangerlookup_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
I20260806 22:41:44.049440 747 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwWuo9B/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/postgres -l logfile start
2026-08-06 22:41:46.259 UTC [18810] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:41:46.259 UTC [18810] LOG: listening on IPv4 address "127.0.186.212", port 60937
2026-08-06 22:41:46.262 UTC [18810] LOG: listening on Unix socket "/tmp/.s.PGSQL.60937"
2026-08-06 22:41:46.289 UTC [18813] LOG: database system was shut down at 2026-08-06 22:41:45 UTC
2026-08-06 22:41:46.293 UTC [18810] LOG: database system is ready to accept connections
I20260806 22:41:47.327713 747 mini_postgres.cc:96] Postgres bound to 60937
2026-08-06 22:41:47.334 UTC [18827] FATAL: database "slave" does not exist
127.0.186.212:60937 - accepting connections
I20260806 22:41:47.334890 747 mini_ranger.cc:161] Starting Ranger...
I20260806 22:41:47.349828 747 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:41:47.727210 747 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:41:47.727305 747 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin
2026-08-06 22:41:48,006 [I] DB FLAVOR :POSTGRES
2026-08-06 22:41:48,006 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:41:48,006 [I] Checking connection..
2026-08-06 22:41:48,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:41:48,246 [I] Checking connection passed.
2026-08-06 22:41:48,246 [I] --------- Verifying version history table ---------
2026-08-06 22:41:48,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:48,502 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:41:48,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:48,742 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:41:48,742 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:41:48,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:41:48.994 UTC [18921] WARNING: there is no transaction in progress
2026-08-06 22:41:49,000 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:41:49,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:41:49,247 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:41:49,247 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:41:49,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:41:49,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:41:49,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-06 22:41:49,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:41:50,187 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:41:50,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:41:50,449 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:41:50,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:41:50,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:41:50,946 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:41:50,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:41:51.833 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.849 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.864 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.879 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.897 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.917 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.979 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.985 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:51.999 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.009 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.019 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.033 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.043 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.049 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.055 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52.062 UTC [19072] WARNING: there is no transaction in progress
2026-08-06 22:41:52,642 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:41:52,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:41:52,906 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:41:52,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:41:53,137 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:41:53,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:41:53,383 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:41:53,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:41:53,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:41:53,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-06 22:41:54,130 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:41:54,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:41:54,370 [I] DB_PATCHES have already been applied
I20260806 22:41:54.376008 747 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwWuo9B/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:41:54.376071 747 mini_ranger.cc:193] Using host: 127.0.186.212
I20260806 22:41:54.379680 747 mini_ranger.cc:227] Ranger admin URL: http://127.0.186.212:60777
Aug 06, 2026 10:41:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:41:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:41:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:41:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:41:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:41:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:41:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:41:55.565328 747 mini_ranger.cc:160] Time spent starting Ranger: real 8.230s user 0.002s sys 0.007s
Aug 06, 2026 10:42:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:42:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:42:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:42:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:42:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:42:17.267123 747 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangerkms_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/rangerkms_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.186.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.186.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.186.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/HTTP_127.0.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:42:17 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056137, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:42:17.365968 747 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:42:17.376538 747 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:42:18.136749 747 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:42:18.136834 747 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms
I20260806 22:42:18.136849 747 mini_ranger_kms.cc:227] Using postgres at 127.0.186.212:60937
2026-08-06 22:42:18,288 [I] DB FLAVOR :POSTGRES
2026-08-06 22:42:18,289 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:42:18,289 [I] Checking connection
2026-08-06 22:42:18,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:42:18,514 [I] connection success
2026-08-06 22:42:18,515 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:42:18,515 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:42:18,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:42:18,770 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:42:18,770 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:42:18,770 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:42:18,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.186.212:60937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:42:19,022 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:42:19.468144 747 mini_ranger_kms.cc:310] Created kms service
I20260806 22:42:19.665226 747 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:42:19.710382 747 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:42:19 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056137, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:20.078007 747 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:42:20.078122 747 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwWuo9B/build/release/bin/postgresql.jar:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwWuo9B/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:42:20.078141 747 mini_ranger_kms.cc:240] Using host: 127.0.186.212
I20260806 22:42:20.082706 747 mini_ranger_kms.cc:276] Ranger KMS PID: 19329
I20260806 22:42:20.082770 747 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.186.212:56103
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:42:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:42:22 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:22.544679 747 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.179s user 0.011s sys 0.125s
I20260806 22:42:22.544775 747 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:42:22.544824 747 mini_ranger_kms.cc:401] 127.0.186.212:56103/kms/v1/keys
Aug 06 22:42:22 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
Aug 06 22:42:23 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.186.212@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
Aug 06 22:42:25 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:42:25.427757 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36249
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:36249
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:25.510972 19412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:25.511174 19412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:25.511195 19412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:25.512357 19412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:25.512399 19412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:25.512413 19412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:25.512424 19412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:25.512434 19412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:25.513978 19412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.186.212:56103/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:36249
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:25.514205 19412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:25.514417 19412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:25.518915 19422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:25.518997 19424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:25.519140 19412 server_base.cc:1034] running on GCE node
W20260806 22:42:25.518947 19421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:25.519466 19412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:25.519739 19412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:25.520877 19412 hybrid_clock.cc:648] HybridClock initialized: now 1786056145520864 us; error 30 us; skew 500 ppm
Aug 06 22:42:25 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:25.523190 19412 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:25.523630 19412 webserver.cc:460] Webserver started at http://127.0.186.254:43983/ using document root <none> and password file <none>
I20260806 22:42:25.523838 19412 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:25.523877 19412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:25.524000 19412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:42:25 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:25.574124 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-gjrz"
server_key: "a864e6de072c0adc43fb9850c8fa26f4"
server_key_iv: "05e5dca7921bce8a0efdf16929b1ddeb"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:25.574501 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-gjrz"
server_key: "a864e6de072c0adc43fb9850c8fa26f4"
server_key_iv: "05e5dca7921bce8a0efdf16929b1ddeb"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:25.575947 19412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:25.591068 19433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:25.591430 19412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:25.591539 19412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-gjrz"
server_key: "a864e6de072c0adc43fb9850c8fa26f4"
server_key_iv: "05e5dca7921bce8a0efdf16929b1ddeb"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:25.591634 19412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:25.610455 19412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:25.612365 19412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:25.612512 19412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:25.617004 19412 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:36249
I20260806 22:42:25.617074 19522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:36249 every 8 connection(s)
I20260806 22:42:25.617421 19412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:25.618209 19523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:25.621392 19523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Bootstrap starting.
I20260806 22:42:25.622117 19523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:25.622463 19523 log.cc:824] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:25.623098 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 19412
I20260806 22:42:25.623188 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:42:25.623241 19523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: No bootstrap required, opened a new log
I20260806 22:42:25.623406 747 external_mini_cluster.cc:1428] Setting key 824eccf42d0620f669d1b27ae2d00cde
I20260806 22:42:25.626197 19523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:25.626380 19523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:25.626422 19523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6695fa4cda3c4d5ba47f7fa97f262dc3, State: Initialized, Role: FOLLOWER
I20260806 22:42:25.626566 19523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:25.626647 19523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:25.626677 19523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:25.626719 19523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:25.627457 19523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:25.627583 19523 leader_election.cc:302] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6695fa4cda3c4d5ba47f7fa97f262dc3; no voters:
I20260806 22:42:25.627753 19523 leader_election.cc:288] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:25.627875 19529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:25.628053 19529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 LEADER]: Becoming Leader. State: Replica: 6695fa4cda3c4d5ba47f7fa97f262dc3, State: Running, Role: LEADER
I20260806 22:42:25.628090 19523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:25.628214 19529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:25.628610 19529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } } }
I20260806 22:42:25.628752 19529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:25.628923 19529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6695fa4cda3c4d5ba47f7fa97f262dc3. Latest consensus state: current_term: 1 leader_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } } }
I20260806 22:42:25.629019 19529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:25.629575 19523 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 22:42:25 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:25.632084 19539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:25.634296 19539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:25.641219 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:25.623972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.0.186.254:36249)
0806 22:42:25.624229 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:25.624237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:25.624259 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:42:25.626153 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:25.626160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:25.626209 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:25.626332 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:25.627036 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:25.628271 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:25.629332 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:25.629781 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:25.629876 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:25.632330 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:25.632371 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:25.632371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:25.632468 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:25.635253 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:25.639735 (+ 4482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:25.639739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:25.639739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:25.639866 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:25.640085 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:25.640086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:25.640086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:25.640404 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:25.640888 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:25.640944 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:42:25.641058 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":60,"threads_started":1}
I20260806 22:42:25.644711 19539 catalog_manager.cc:1347] Generated new cluster ID: f894ea12a2174dbb89967f9a51e66900
I20260806 22:42:25.644757 19539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:25.649662 19539 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:25.650384 19539 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:25.658838 19539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Generated new TSK 0
I20260806 22:42:25.659001 19539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:42:26.326277 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.422171 19586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.422438 19586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.422472 19586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.424163 19586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.424219 19586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.424256 19586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:26.425839 19586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:26.426062 19586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.426288 19586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.431205 19593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.431217 19596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.431322 19594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.431457 19586 server_base.cc:1034] running on GCE node
I20260806 22:42:26.431761 19586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.432075 19586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.433257 19586 hybrid_clock.cc:648] HybridClock initialized: now 1786056146433239 us; error 33 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.436336 19586 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:26.436853 19586 webserver.cc:460] Webserver started at http://127.0.186.193:35227/ using document root <none> and password file <none>
I20260806 22:42:26.437124 19586 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.437188 19586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.437317 19586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:26.454562 19586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "6a6d3f2888d020d7e0dca304f1ff6a26"
server_key_iv: "647fb2811c0923d99f9eab84b0d1ff7f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.454967 19586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "6a6d3f2888d020d7e0dca304f1ff6a26"
server_key_iv: "647fb2811c0923d99f9eab84b0d1ff7f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.456779 19586 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:26.472484 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.472715 19586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:26.472862 19586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "6a6d3f2888d020d7e0dca304f1ff6a26"
server_key_iv: "647fb2811c0923d99f9eab84b0d1ff7f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.473042 19586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:26.489050 19586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:26.491003 19586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:26.491144 19586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:26.491357 19586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:26.491674 19586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:26.491707 19586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.491736 19586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:26.491761 19586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.497722 19586 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:40351
I20260806 22:42:26.498188 19586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:26.498308 19793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:40351 every 8 connection(s)
I20260806 22:42:26.504206 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 19586
I20260806 22:42:26.504307 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:26.504493 747 external_mini_cluster.cc:1428] Setting key 40471502a2fa0afdcaf6892edbd5400c
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:26.510008 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.499213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.193:40299)
0806 22:42:26.499394 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:26.499445 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:26.500119 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:26.500551 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:26.500556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:26.500808 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:26.501109 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.501117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.502161 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.502162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.503457 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.503461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.503979 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.503980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.504132 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.504700 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:26.504711 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:26.506293 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:26.508352 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.508360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.508362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.508641 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.508876 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.508877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.508877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.509013 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.509321 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:26.509330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:26.509500 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:42:26.509618 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:42:26.509776 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":111,"threads_started":1}
I20260806 22:42:26.510373 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.499540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:40299 (local address 127.0.186.254:36249)
0806 22:42:26.500309 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.500311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.500319 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:26.500346 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.500346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.500360 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.500430 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.501222 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.502054 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.503540 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.503872 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.503939 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.506412 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.506413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.506413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.506421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.508267 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.508730 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.508731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.508732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.508806 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.509090 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.509091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.509092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.509239 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.509313 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.510164 (+ 851us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.510278 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":86,"threads_started":1}
I20260806 22:42:26.510928 19795 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:26.511034 19795 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:26.511219 19795 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:26.511848 19459 ts_manager.cc:194] Registered new tserver with Master: ca33ae6d8a99448b9f567d8e9fa27cf4 (127.0.186.193:40351)
I20260806 22:42:26.512708 19459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:40299
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:42:26.534010 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.638908 19804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.639106 19804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.639137 19804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.640321 19804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.640362 19804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.640395 19804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:26.642205 19804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:26.642431 19804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.642686 19804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.647399 19820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.647393 19818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.647393 19815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.648159 19804 server_base.cc:1034] running on GCE node
I20260806 22:42:26.648483 19804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.648726 19804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.649875 19804 hybrid_clock.cc:648] HybridClock initialized: now 1786056146649856 us; error 32 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.652663 19804 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:26.653077 19804 webserver.cc:460] Webserver started at http://127.0.186.194:32955/ using document root <none> and password file <none>
I20260806 22:42:26.653283 19804 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.653330 19804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.653455 19804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.675590 19804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "eddc0891c93a49f2972b6696d6d33711"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "05e07744afe88d46c25b13a59cc3ea29"
server_key_iv: "dc056e432b4e6743c65f85416274596f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.676035 19804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "eddc0891c93a49f2972b6696d6d33711"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "05e07744afe88d46c25b13a59cc3ea29"
server_key_iv: "dc056e432b4e6743c65f85416274596f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.681514 19804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:42:26.701436 19832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.701726 19804 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260806 22:42:26.701902 19804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "eddc0891c93a49f2972b6696d6d33711"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "05e07744afe88d46c25b13a59cc3ea29"
server_key_iv: "dc056e432b4e6743c65f85416274596f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.702083 19804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:26.721853 19804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:26.723728 19804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:26.723863 19804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:26.724056 19804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:26.724385 19804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:26.724422 19804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.724452 19804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:26.724489 19804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.730799 19804 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34949
I20260806 22:42:26.731312 19804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:26.741374 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 19804
I20260806 22:42:26.741487 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:26.741690 747 external_mini_cluster.cc:1428] Setting key 2fca5d6e85c2a76ce871398fb6e9c003
I20260806 22:42:26.743625 19995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34949 every 8 connection(s)
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
I20260806 22:42:26.762338 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.744300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.194:36185)
0806 22:42:26.744729 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:26.744777 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:26.745272 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:26.745586 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:26.745591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:26.745838 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:26.746114 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.746121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.747068 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.747069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.748326 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:26.748330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.748890 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.748891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:26.749033 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.749617 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:26.749629 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:26.757783 (+ 8154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:26.759720 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.759728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.759731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.760043 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.761044 (+ 1001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:26.761045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:26.761045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:26.761223 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:26.761549 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:26.761557 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:26.761742 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 22:42:26.761888 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:42:26.762084 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":107,"threads_started":1}
I20260806 22:42:26.762810 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:26.744505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:36185 (local address 127.0.186.254:36249)
0806 22:42:26.744572 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:26.744574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:26.744872 (+ 298us) server_negotiation.cc:407] Connection header received
0806 22:42:26.745371 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:26.745373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:26.745387 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:26.745438 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:26.746224 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.746958 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.748433 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:26.748733 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:26.748806 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:26.757914 (+ 9108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:26.757917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:26.757917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:26.757927 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:26.759622 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.760146 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.760147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.760148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.760213 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:26.761311 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:26.761312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:26.761312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:26.761460 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:26.761521 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:26.762612 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0806 22:42:26.762716 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:42:26.763291 19997 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:26.763399 19997 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:26.763618 19997 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:26.764037 19459 ts_manager.cc:194] Registered new tserver with Master: eddc0891c93a49f2972b6696d6d33711 (127.0.186.194:34949)
I20260806 22:42:26.764483 19459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:36185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:42:26.781605 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:26.908676 20006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:26.909062 20006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:26.909162 20006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:26.910864 20006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:26.911094 20006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:26.911194 20006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:26.913365 20006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:26.913806 20006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:26.914145 20006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:26.921444 20014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.921804 20013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:26.922081 20024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:26.922863 20006 server_base.cc:1034] running on GCE node
I20260806 22:42:26.923240 20006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:26.923583 20006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:26.924738 20006 hybrid_clock.cc:648] HybridClock initialized: now 1786056146924710 us; error 41 us; skew 500 ppm
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:26.928119 20006 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:26.928663 20006 webserver.cc:460] Webserver started at http://127.0.186.195:36491/ using document root <none> and password file <none>
I20260806 22:42:26.929034 20006 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:26.929160 20006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:26.929344 20006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:42:26 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:26.953008 20006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "02765bf5b21c4c18a8f260cc6329371a"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "5e2a106981b404a1b3b3b1d3611c0fb7"
server_key_iv: "dbcfc9eeef05a61032c79b332ddc6f72"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.953827 20006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "02765bf5b21c4c18a8f260cc6329371a"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "5e2a106981b404a1b3b3b1d3611c0fb7"
server_key_iv: "dbcfc9eeef05a61032c79b332ddc6f72"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.959663 20006 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 22:42:26.993414 20035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:26.994284 20006 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.000s
I20260806 22:42:26.994407 20006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "02765bf5b21c4c18a8f260cc6329371a"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "5e2a106981b404a1b3b3b1d3611c0fb7"
server_key_iv: "dbcfc9eeef05a61032c79b332ddc6f72"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:26.994514 20006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.007148 20006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.009598 20006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.009821 20006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.010129 20006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:27.010581 20006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:27.010684 20006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.010756 20006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:27.012348 20006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 22:42:27.030020 20006 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35591
I20260806 22:42:27.030607 20006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:27.032657 20195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35591 every 8 connection(s)
I20260806 22:42:27.033425 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 20006
I20260806 22:42:27.033520 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:27.033711 747 external_mini_cluster.cc:1428] Setting key 74003a43ab9e2e8b99999bf94b36259d
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:27.044363 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.033216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:33685 (local address 127.0.186.254:36249)
0806 22:42:27.033853 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.033855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.034653 (+ 798us) server_negotiation.cc:407] Connection header received
0806 22:42:27.035336 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.035338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.035351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.035382 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.036273 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.036995 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.038359 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.038632 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.038672 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.041274 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.041275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.041275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.041282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.042853 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.043332 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.043333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.043334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.043394 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.043657 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.043658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.043658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.043782 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.043828 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.044161 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.044269 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608}
I20260806 22:42:27.045722 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.032871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.195:33685)
0806 22:42:27.033090 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:27.033139 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:27.035188 (+ 2049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:27.035606 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:27.035610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:27.035881 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:27.036167 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.036174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.037108 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.037110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.038262 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.038265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.038758 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.038759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.038897 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.039520 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:27.039531 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:27.041167 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:27.042947 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.042953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.042955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.043226 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.043470 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.043470 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.043471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.043592 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.043858 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:27.043863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:27.044017 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:42:27.045316 (+ 1299us) client_negotiation.cc:240] Negotiation successful
0806 22:42:27.045496 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":115,"threads_started":1}
I20260806 22:42:27.046334 20196 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:27.046433 20196 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.046605 20196 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:27.046974 19459 ts_manager.cc:194] Registered new tserver with Master: 02765bf5b21c4c18a8f260cc6329371a (127.0.186.195:35591)
I20260806 22:42:27.047463 19459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:33685
I20260806 22:42:27.056001 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:27.056782 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 19586
I20260806 22:42:27.147812 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 19804
I20260806 22:42:27.189316 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 20006
I20260806 22:42:27.206374 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 19412
I20260806 22:42:27.218653 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36249
--webserver_interface=127.0.186.254
--webserver_port=43983
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:36249
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:27.324168 20201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.324365 20201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.324386 20201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.325541 20201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:27.325588 20201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.325609 20201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.325625 20201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:27.325641 20201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:27.327482 20201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:36249
--ranger_config_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:36249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:27.327721 20201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.327946 20201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.332558 20210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.332614 20213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.332859 20201 server_base.cc:1034] running on GCE node
W20260806 22:42:27.332558 20211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.333170 20201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.333461 20201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.334621 20201 hybrid_clock.cc:648] HybridClock initialized: now 1786056147334603 us; error 30 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.337227 20201 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:27.337680 20201 webserver.cc:460] Webserver started at http://127.0.186.254:43983/ using document root <none> and password file <none>
I20260806 22:42:27.337877 20201 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.337934 20201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:27.374229 20201 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:27.374992 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.375165 20201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:27.375281 20201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3"
format_stamp: "Formatted at 2026-08-06 22:42:25 on dist-test-slave-gjrz"
server_key: "a864e6de072c0adc43fb9850c8fa26f4"
server_key_iv: "05e5dca7921bce8a0efdf16929b1ddeb"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:27.375651 20201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.401095 20201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.402988 20201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.403131 20201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.407022 20201 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:36249
I20260806 22:42:27.407091 20306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:36249 every 8 connection(s)
I20260806 22:42:27.407436 20201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:27.408672 20310 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:27.409322 20310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Bootstrap starting.
I20260806 22:42:27.415252 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 20201
I20260806 22:42:27.415521 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:40351
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:36249
--webserver_port=35227
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:27.417151 20310 log.cc:824] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:27.418684 20310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:27.418918 20310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Bootstrap complete.
I20260806 22:42:27.421072 20310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:27.421305 20310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6695fa4cda3c4d5ba47f7fa97f262dc3, State: Initialized, Role: FOLLOWER
I20260806 22:42:27.421463 20310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:27.421538 20310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:27.421595 20310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:27.421643 20310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:42:27.422628 20310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:27.422730 20310 leader_election.cc:302] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6695fa4cda3c4d5ba47f7fa97f262dc3; no voters:
I20260806 22:42:27.422889 20310 leader_election.cc:288] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:42:27.422935 20317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:42:27.423085 20317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [term 2 LEADER]: Becoming Leader. State: Replica: 6695fa4cda3c4d5ba47f7fa97f262dc3, State: Running, Role: LEADER
I20260806 22:42:27.423154 20310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:27.423285 20317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } }
I20260806 22:42:27.423575 20319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } } }
I20260806 22:42:27.423650 20319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:27.423995 20331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:27.424029 20320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6695fa4cda3c4d5ba47f7fa97f262dc3. Latest consensus state: current_term: 2 leader_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6695fa4cda3c4d5ba47f7fa97f262dc3" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 36249 } } }
I20260806 22:42:27.424144 20320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:27.424655 20331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:27.424839 20331 catalog_manager.cc:1259] Loaded cluster ID: f894ea12a2174dbb89967f9a51e66900
I20260806 22:42:27.424916 20331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:27.424984 20310 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:42:27.426218 20331 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:27.426403 20331 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Loaded TSK: 0
I20260806 22:42:27.426616 20331 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:42:27.529647 20313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.529987 20313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.530045 20313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.531685 20313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.531778 20313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.531849 20313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:27.533977 20313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:40351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=35227
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:27.534222 20313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.534524 20313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.540903 20365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.540889 20364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.540889 20367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.541747 20313 server_base.cc:1034] running on GCE node
I20260806 22:42:27.541983 20313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.542253 20313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.543412 20313 hybrid_clock.cc:648] HybridClock initialized: now 1786056147543395 us; error 30 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.546887 20313 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:27.547446 20313 webserver.cc:460] Webserver started at http://127.0.186.193:35227/ using document root <none> and password file <none>
I20260806 22:42:27.547757 20313 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.547887 20313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:27.566004 20313 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:27.566843 20377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.567042 20313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:27.567126 20313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "6a6d3f2888d020d7e0dca304f1ff6a26"
server_key_iv: "647fb2811c0923d99f9eab84b0d1ff7f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:27.567586 20313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.580439 20313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.582688 20313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.582864 20313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.583105 20313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:27.583494 20313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:27.583576 20313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.583624 20313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:27.583657 20313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.589521 20313 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:40351
I20260806 22:42:27.589993 20313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:27.592501 20556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:40351 every 8 connection(s)
I20260806 22:42:27.595538 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 20313
I20260806 22:42:27.595845 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:34949
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:36249
--webserver_port=32955
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:27.608083 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.593122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.193:33601)
0806 22:42:27.593297 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:27.593349 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:27.593968 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:27.594678 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:27.594682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:27.594935 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:27.595228 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.595236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.599304 (+ 4068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.599305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.600590 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.600594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.601579 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.601580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.601717 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.602338 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:27.602349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:27.604004 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:27.606173 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.606180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.606183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.606467 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.606761 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.606762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.606762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.606895 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.607430 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:27.607437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:27.607588 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:42:27.607690 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:42:27.607843 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260806 22:42:27.608606 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.593226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:33601 (local address 127.0.186.254:36249)
0806 22:42:27.593446 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.593455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.593497 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:42:27.594285 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.594293 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.594355 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.594494 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.597705 (+ 3211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.599141 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.600693 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.601352 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.601463 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.604128 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.604132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.604132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.604145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.606078 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.606564 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.606568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.606568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.606687 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.606974 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.606975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.606976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.607319 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.607399 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.608284 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.608406 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260806 22:42:27.611050 20559 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:27.611147 20559 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.611354 20559 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
W20260806 22:42:27.612290 20559 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36249 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:42:27.612356 20559 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.612417 20559 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:27.612824 20559 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.612893 20559 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
W20260806 22:42:27.613243 20559 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:42:27.726502 20563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.726768 20563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.726804 20563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.728482 20563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.728541 20563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.728592 20563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:27.730767 20563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:34949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=32955
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:27.731035 20563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.731341 20563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.736537 20575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.736537 20572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.736760 20563 server_base.cc:1034] running on GCE node
W20260806 22:42:27.736732 20573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.737030 20563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.737236 20563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.738372 20563 hybrid_clock.cc:648] HybridClock initialized: now 1786056147738365 us; error 40 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.740916 20563 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:27.741328 20563 webserver.cc:460] Webserver started at http://127.0.186.194:32955/ using document root <none> and password file <none>
I20260806 22:42:27.741544 20563 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.741607 20563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:27.757589 20563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:27.758358 20593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.758539 20563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:42:27.758620 20563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "eddc0891c93a49f2972b6696d6d33711"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "05e07744afe88d46c25b13a59cc3ea29"
server_key_iv: "dc056e432b4e6743c65f85416274596f"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:27.759057 20563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.778023 20563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.779815 20563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.779946 20563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.780150 20563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:27.780460 20563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:27.780501 20563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.780530 20563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:27.780553 20563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.786755 20563 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34949
I20260806 22:42:27.787130 20761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34949 every 8 connection(s)
I20260806 22:42:27.788127 20563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:27.793140 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 20563
I20260806 22:42:27.793498 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.186.212:56103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:35591
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:36249
--webserver_port=36491
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:27.801473 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.789096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:49865 (local address 127.0.186.254:36249)
0806 22:42:27.789156 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.789158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.789422 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:42:27.790063 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.790064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.790080 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.790142 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.790945 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.791746 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.793213 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.793508 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.793557 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.796814 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.796815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.796816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.796824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.800003 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.800476 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.800478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.800478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.800542 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.800822 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.800823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.800824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.800952 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.800996 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.801287 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.801380 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:42:27.801622 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.789098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.194:49865)
0806 22:42:27.789268 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:27.789335 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:27.789942 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:27.790274 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:27.790279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:27.790533 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:27.790815 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.790823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.791850 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.791851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.793101 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.793105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.793672 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.793672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.793802 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.794321 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:27.794332 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:27.796706 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:27.800098 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.800105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.800107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.800402 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.800602 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.800602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.800603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.800735 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.801035 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:27.801041 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:27.801159 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 22:42:27.801244 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:42:27.801389 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260806 22:42:27.802086 20765 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:27.802191 20765 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.802371 20765 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
W20260806 22:42:27.803002 20765 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36249 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:42:27.803069 20765 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.803135 20765 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:27.803464 20765 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.803550 20765 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
W20260806 22:42:27.803822 20765 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:42:27.915903 20769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:27.916178 20769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:27.916222 20769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:27.917914 20769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:27.917971 20769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:27.918008 20769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:27.920123 20769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.186.212:56103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:35591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=36491
--tserver_master_addrs=127.0.186.254:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:27.920363 20769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:27.920640 20769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:27.926396 20780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.926369 20784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:27.926352 20782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:27.926610 20769 server_base.cc:1034] running on GCE node
I20260806 22:42:27.926856 20769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:27.927125 20769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:27.928258 20769 hybrid_clock.cc:648] HybridClock initialized: now 1786056147928232 us; error 41 us; skew 500 ppm
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:27.936839 20769 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:27.937451 20769 webserver.cc:460] Webserver started at http://127.0.186.195:36491/ using document root <none> and password file <none>
I20260806 22:42:27.937783 20769 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:27.937912 20769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:27.949486 20769 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:27.950184 20797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.950374 20769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:27.950454 20769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "02765bf5b21c4c18a8f260cc6329371a"
format_stamp: "Formatted at 2026-08-06 22:42:26 on dist-test-slave-gjrz"
server_key: "5e2a106981b404a1b3b3b1d3611c0fb7"
server_key_iv: "dbcfc9eeef05a61032c79b332ddc6f72"
server_key_version: "kuduclusterkey@0"
I20260806 22:42:27.950842 20769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:27.968873 20769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:27.971110 20769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:27.971227 20769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:27.971503 20769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:27.971824 20769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:27.971874 20769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.971905 20769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:27.971930 20769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:27.979105 20769 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35591
I20260806 22:42:27.979171 20965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35591 every 8 connection(s)
I20260806 22:42:27.979722 20769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:27.981271 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 20769
Aug 06 22:42:27 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:27.993669 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.980481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:34217 (local address 127.0.186.254:36249)
0806 22:42:27.980548 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.980550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.980707 (+ 157us) server_negotiation.cc:407] Connection header received
0806 22:42:27.981352 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.981354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.981367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.981414 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.982089 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.982605 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.984183 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.984395 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.984454 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.990239 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.990241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.990241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.990248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.991971 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.992560 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.992562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.992562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.992641 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.992950 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.992951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.992951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.993089 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.993155 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.993495 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.993573 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:27.993894 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.980413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:36249 (local address 127.0.186.195:34217)
0806 22:42:27.980566 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:27.980616 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:27.981247 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:27.981500 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:27.981504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:27.981693 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:27.981992 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.981998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.983153 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.983154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.984110 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:27.984113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.984531 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.984532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:27.984670 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.985421 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:27.985431 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:27.990131 (+ 4700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:27.992069 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.992076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.992086 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.992465 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.992729 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:27.992730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:27.992730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:27.992879 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:27.993165 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:27.993172 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:27.993347 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:42:27.993481 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 22:42:27.993658 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260806 22:42:27.994582 20967 heartbeater.cc:347] Connected to a master server at 127.0.186.254:36249
I20260806 22:42:27.994678 20967 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.994861 20967 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:27.994870 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:27.981627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.0.186.254:36249)
0806 22:42:27.981977 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:27.981979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:27.981985 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:42:27.982008 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:27.982009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:27.982019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:27.982091 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:27.982845 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.983576 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.984907 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:27.985231 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:27.985297 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:27.986943 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:27.986945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:27.986945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:27.986952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:27.990989 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.992568 (+ 1579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.992570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.992570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.992671 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:27.993179 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:27.993179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:27.993180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:27.993336 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:27.994647 (+ 1311us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:27.994687 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:42:27.994772 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":77,"threads_started":1}
W20260806 22:42:27.995800 20967 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:36249 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:42:27.995864 20967 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.995919 20967 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:27.996291 20967 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:27.996366 20967 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
W20260806 22:42:27.996721 20967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:42:28 dist-test-slave-gjrz krb5kdc[18769](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:28.613432 20559 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.613600 20559 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:28.614024 20249 ts_manager.cc:194] Registered new tserver with Master: ca33ae6d8a99448b9f567d8e9fa27cf4 (127.0.186.193:40351)
I20260806 22:42:28.615031 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:33601
I20260806 22:42:28.803983 20765 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.804145 20765 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:28.804407 20249 ts_manager.cc:194] Registered new tserver with Master: eddc0891c93a49f2972b6696d6d33711 (127.0.186.194:34949)
I20260806 22:42:28.804942 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:49865
Aug 06 22:42:28 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:28.996910 20967 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:28.997054 20967 heartbeater.cc:511] Master 127.0.186.254:36249 requested a full tablet report, sending...
I20260806 22:42:28.997331 20249 ts_manager.cc:194] Registered new tserver with Master: 02765bf5b21c4c18a8f260cc6329371a (127.0.186.195:35591)
I20260806 22:42:28.997722 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:34217
I20260806 22:42:29.001235 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:29.008208 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.002352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.0.186.254:36249)
0806 22:42:29.002470 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.002472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.002591 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:42:29.002648 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.002649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.002663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.002723 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:29.003240 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.003764 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.004443 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.004653 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.004707 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.005841 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:29.005843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:29.005843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:29.005851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:29.007174 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:29.007503 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:29.007504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:29.007505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:29.007558 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:29.007724 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:29.007724 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:29.007725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:29.007828 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:29.007927 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.008027 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.008110 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260806 22:42:29.010035 20249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:29.011742 20249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:29.018172 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40351 (local address 127.0.0.1:40660)
0806 22:42:29.014940 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.014974 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.015026 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.015255 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.015259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.015280 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:29.015350 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.015352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.016315 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.016316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.017098 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.017100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.017879 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.017880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.017961 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.017976 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.018035 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.018093 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":87,"threads_started":1}
I20260806 22:42:29.018178 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.0.186.193:40351)
0806 22:42:29.015122 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.015125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.015133 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:29.015159 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.015160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.015172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.015234 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:29.015429 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.016241 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.017179 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.017806 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.017868 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.017935 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.018058 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.018105 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":65,"threads_started":1}
I20260806 22:42:29.019169 20460 tablet_service.cc:1467] Processing CreateTablet for tablet 3e449ff0849f4c26a87b1e0c9513ecb6 (DEFAULT_TABLE table=test-table [id=a132256b9d1d4ed7b4a364aea4919121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.019553 20460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e449ff0849f4c26a87b1e0c9513ecb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.022225 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34949 (local address 127.0.0.1:60070)
0806 22:42:29.018480 (+ 4196us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.018491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.018538 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.018754 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.018755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.018756 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:29.018787 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.018788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.020116 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.020117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.021099 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.021102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.022034 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.022036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.022094 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.022098 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.022126 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.022161 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4186,"thread_start_us":83,"threads_started":1}
I20260806 22:42:29.022624 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.0.186.194:34949)
0806 22:42:29.015384 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.015389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.018605 (+ 3216us) server_negotiation.cc:407] Connection header received
0806 22:42:29.018643 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.018645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.018662 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.018847 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:29.018918 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.020027 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.021197 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.021933 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.022387 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.022467 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.022494 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.022533 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":53,"threads_started":1}
I20260806 22:42:29.023394 20660 tablet_service.cc:1467] Processing CreateTablet for tablet 3e449ff0849f4c26a87b1e0c9513ecb6 (DEFAULT_TABLE table=test-table [id=a132256b9d1d4ed7b4a364aea4919121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.023753 20660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e449ff0849f4c26a87b1e0c9513ecb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.027424 21024 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Bootstrap starting.
I20260806 22:42:29.028049 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35591 (local address 127.0.0.1:55552)
0806 22:42:29.019333 (+ 4821us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.019345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.019414 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.024311 (+ 4897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.024312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.024315 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:29.024354 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.024355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.025567 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.025569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.026919 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.026922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.027782 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.027783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.027843 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.027847 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.027940 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.027981 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4803,"thread_start_us":80,"threads_started":1}
I20260806 22:42:29.028071 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.014778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.0.186.195:35591)
0806 22:42:29.014874 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.014879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.023995 (+ 9116us) server_negotiation.cc:407] Connection header received
0806 22:42:29.024072 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.024075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.024102 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.024227 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:29.024428 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.025464 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.027025 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.027706 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.027772 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.027834 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.027961 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.027998 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"thread_start_us":46,"threads_started":1}
I20260806 22:42:29.028151 21024 tablet_bootstrap.cc:654] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.028440 21024 log.cc:824] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.028987 20862 tablet_service.cc:1467] Processing CreateTablet for tablet 3e449ff0849f4c26a87b1e0c9513ecb6 (DEFAULT_TABLE table=test-table [id=a132256b9d1d4ed7b4a364aea4919121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:29.029331 20862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e449ff0849f4c26a87b1e0c9513ecb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:29.031837 21027 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Bootstrap starting.
I20260806 22:42:29.032544 21027 tablet_bootstrap.cc:654] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.032825 21027 log.cc:824] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.033669 21027 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: No bootstrap required, opened a new log
I20260806 22:42:29.033735 21027 ts_tablet_manager.cc:1397] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:29.035418 21025 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Bootstrap starting.
I20260806 22:42:29.035686 21027 raft_consensus.cc:348] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.035817 21027 raft_consensus.cc:374] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.035851 21027 raft_consensus.cc:729] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02765bf5b21c4c18a8f260cc6329371a, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.036005 21027 consensus_queue.cc:260] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.036267 21027 ts_tablet_manager.cc:1428] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:29.036346 21025 tablet_bootstrap.cc:654] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:29.036391 20967 heartbeater.cc:503] Master 127.0.186.254:36249 was elected leader, sending a full tablet report...
I20260806 22:42:29.036757 21025 log.cc:824] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:29.038512 21025 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: No bootstrap required, opened a new log
I20260806 22:42:29.038823 21025 ts_tablet_manager.cc:1397] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:29.039659 21024 tablet_bootstrap.cc:492] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: No bootstrap required, opened a new log
I20260806 22:42:29.039743 21024 ts_tablet_manager.cc:1397] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260806 22:42:29.041807 21025 raft_consensus.cc:348] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.041962 21025 raft_consensus.cc:374] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.041996 21025 raft_consensus.cc:729] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eddc0891c93a49f2972b6696d6d33711, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.042129 21025 consensus_queue.cc:260] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.042399 21025 ts_tablet_manager.cc:1428] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:42:29.042430 20765 heartbeater.cc:503] Master 127.0.186.254:36249 was elected leader, sending a full tablet report...
I20260806 22:42:29.042366 21024 raft_consensus.cc:348] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.042529 21024 raft_consensus.cc:374] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:29.042562 21024 raft_consensus.cc:729] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca33ae6d8a99448b9f567d8e9fa27cf4, State: Initialized, Role: FOLLOWER
I20260806 22:42:29.042670 21024 consensus_queue.cc:260] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.042975 21024 ts_tablet_manager.cc:1428] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:42:29.043064 20559 heartbeater.cc:503] Master 127.0.186.254:36249 was elected leader, sending a full tablet report...
W20260806 22:42:29.091375 20560 tablet.cc:2369] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:42:29 dist-test-slave-gjrz krb5kdc[18769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for HTTP/127.0.186.212@KRBTEST.COM
I20260806 22:42:29.209565 21037 raft_consensus.cc:482] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:29.209754 21037 raft_consensus.cc:504] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.210170 21037 leader_election.cc:288] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca33ae6d8a99448b9f567d8e9fa27cf4 (127.0.186.193:40351), 02765bf5b21c4c18a8f260cc6329371a (127.0.186.195:35591)
I20260806 22:42:29.218760 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.210925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:57523 (local address 127.0.186.193:40351)
0806 22:42:29.210972 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.210974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.211553 (+ 579us) server_negotiation.cc:407] Connection header received
0806 22:42:29.211660 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.211661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.211672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.211701 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:29.212243 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.212995 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.217500 (+ 4505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.218298 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.218347 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.218357 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.218640 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.218687 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:29.219378 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.210471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40351 (local address 127.0.186.194:57523)
0806 22:42:29.211484 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.211496 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.211584 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.211768 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.211769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.211771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:29.212145 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.212148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.213096 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.213097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.214502 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.214505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.218417 (+ 3912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.218418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.218550 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.218555 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.219080 (+ 525us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.219127 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":155,"threads_started":1}
I20260806 22:42:29.219899 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e449ff0849f4c26a87b1e0c9513ecb6" candidate_uuid: "eddc0891c93a49f2972b6696d6d33711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" is_pre_election: true
I20260806 22:42:29.220088 20490 raft_consensus.cc:2454] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eddc0891c93a49f2972b6696d6d33711 in term 0.
I20260806 22:42:29.220469 20594 leader_election.cc:302] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca33ae6d8a99448b9f567d8e9fa27cf4, eddc0891c93a49f2972b6696d6d33711; no voters:
I20260806 22:42:29.220607 21037 raft_consensus.cc:2781] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:29.220649 21037 raft_consensus.cc:482] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:29.220670 21037 raft_consensus.cc:3037] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.221114 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.211405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50925 (local address 127.0.186.195:35591)
0806 22:42:29.211455 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.211457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.214756 (+ 3299us) server_negotiation.cc:407] Connection header received
0806 22:42:29.214853 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.214854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.214863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.214890 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:29.215277 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.215994 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.219776 (+ 3782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.220531 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.220579 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.220587 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.221000 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.221049 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:29.221359 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.211102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35591 (local address 127.0.186.194:50925)
0806 22:42:29.214683 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.214694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.214784 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.214958 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:29.214959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:29.214961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:29.215172 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.215174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.216091 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.216092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.217390 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:29.217393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.220770 (+ 3377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.220772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:29.220904 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.220909 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:29.221219 (+ 310us) client_negotiation.cc:240] Negotiation successful
0806 22:42:29.221267 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3537,"thread_start_us":186,"threads_started":1}
I20260806 22:42:29.221666 20894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e449ff0849f4c26a87b1e0c9513ecb6" candidate_uuid: "eddc0891c93a49f2972b6696d6d33711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02765bf5b21c4c18a8f260cc6329371a" is_pre_election: true
I20260806 22:42:29.221825 20894 raft_consensus.cc:2454] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eddc0891c93a49f2972b6696d6d33711 in term 0.
I20260806 22:42:29.222164 21037 raft_consensus.cc:504] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.222302 21037 leader_election.cc:288] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [CANDIDATE]: Term 1 election: Requested vote from peers ca33ae6d8a99448b9f567d8e9fa27cf4 (127.0.186.193:40351), 02765bf5b21c4c18a8f260cc6329371a (127.0.186.195:35591)
I20260806 22:42:29.222445 20894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e449ff0849f4c26a87b1e0c9513ecb6" candidate_uuid: "eddc0891c93a49f2972b6696d6d33711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02765bf5b21c4c18a8f260cc6329371a"
I20260806 22:42:29.222523 20894 raft_consensus.cc:3037] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.222800 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e449ff0849f4c26a87b1e0c9513ecb6" candidate_uuid: "eddc0891c93a49f2972b6696d6d33711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4"
I20260806 22:42:29.222874 20490 raft_consensus.cc:3037] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:29.223838 20894 raft_consensus.cc:2454] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eddc0891c93a49f2972b6696d6d33711 in term 1.
I20260806 22:42:29.224123 20600 leader_election.cc:302] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02765bf5b21c4c18a8f260cc6329371a, eddc0891c93a49f2972b6696d6d33711; no voters:
I20260806 22:42:29.224248 20490 raft_consensus.cc:2454] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eddc0891c93a49f2972b6696d6d33711 in term 1.
I20260806 22:42:29.224530 21037 raft_consensus.cc:2781] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:29.224706 21037 raft_consensus.cc:686] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 1 LEADER]: Becoming Leader. State: Replica: eddc0891c93a49f2972b6696d6d33711, State: Running, Role: LEADER
I20260806 22:42:29.224808 21037 consensus_queue.cc:237] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } }
I20260806 22:42:29.225705 20249 catalog_manager.cc:5495] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 reported cstate change: term changed from 0 to 1, leader changed from <none> to eddc0891c93a49f2972b6696d6d33711 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "eddc0891c93a49f2972b6696d6d33711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eddc0891c93a49f2972b6696d6d33711" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:29.230849 20968 tablet.cc:2369] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:29.247963 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.244583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.0.186.194:34949)
0806 22:42:29.244632 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:29.244634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:29.244642 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:29.244677 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:29.244677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:29.244697 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:29.244764 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:29.244946 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.245677 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.246757 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:29.247041 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:29.247112 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:29.247400 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:29.247532 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:29.247627 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:29.247866 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:42:29.247890 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:29.250643 20490 raft_consensus.cc:1261] T 3e449ff0849f4c26a87b1e0c9513ecb6 P ca33ae6d8a99448b9f567d8e9fa27cf4 [term 1 FOLLOWER]: Refusing update from remote peer eddc0891c93a49f2972b6696d6d33711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:29.250703 20894 raft_consensus.cc:1261] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 1 FOLLOWER]: Refusing update from remote peer eddc0891c93a49f2972b6696d6d33711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:29.251034 21043 consensus_queue.cc:1035] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca33ae6d8a99448b9f567d8e9fa27cf4" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 40351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:29.251217 21037 consensus_queue.cc:1035] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02765bf5b21c4c18a8f260cc6329371a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:29.255030 21050 mvcc.cc:205] Tried to move back new op lower bound from 7315685987327053824 to 7315685987225276416. Current Snapshot: MvccSnapshot[applied={T|T < 7315685987327053824}]
I20260806 22:42:29.255697 21052 mvcc.cc:205] Tried to move back new op lower bound from 7315685987327053824 to 7315685987225276416. Current Snapshot: MvccSnapshot[applied={T|T < 7315685987327053824}]
I20260806 22:42:29.259140 20249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:29.259228 20249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:29.260015 20249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6695fa4cda3c4d5ba47f7fa97f262dc3: Sending DeleteTablet for 3 replicas of tablet 3e449ff0849f4c26a87b1e0c9513ecb6
I20260806 22:42:29.260535 20660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e449ff0849f4c26a87b1e0c9513ecb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:29 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:60070
I20260806 22:42:29.260596 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 20313
I20260806 22:42:29.263185 20862 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e449ff0849f4c26a87b1e0c9513ecb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:29 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:55552
I20260806 22:42:29.264798 21061 tablet_replica.cc:330] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: stopping tablet replica
I20260806 22:42:29.264895 21061 raft_consensus.cc:2229] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:29.264993 21061 raft_consensus.cc:2258] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:42:29.275827 20594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:40351 (error 108)
W20260806 22:42:29.276285 20227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:40351 (error 108)
I20260806 22:42:29.277469 21060 tablet_replica.cc:330] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: stopping tablet replica
I20260806 22:42:29.277704 21060 raft_consensus.cc:2229] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:29.277871 21060 raft_consensus.cc:2258] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:29.278612 21060 ts_tablet_manager.cc:1905] T 3e449ff0849f4c26a87b1e0c9513ecb6 P eddc0891c93a49f2972b6696d6d33711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:29.279625 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.279505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40351 (local address 127.0.186.194:41541)
0806 22:42:29.279553 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.279594 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:40351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 22:42:29.279976 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 20563
I20260806 22:42:29.289011 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:29.288832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:40351 (local address 127.0.0.1:40666)
0806 22:42:29.288909 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.288968 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:40351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 22:42:29.289247 20227 catalog_manager.cc:4584] TS ca33ae6d8a99448b9f567d8e9fa27cf4 (127.0.186.193:40351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e449ff0849f4c26a87b1e0c9513ecb6: Network error: Client connection negotiation failed: client connection to 127.0.186.193:40351: connect: Connection refused (error 111)
I20260806 22:42:29.291492 21061 ts_tablet_manager.cc:1905] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:29.293395 21061 ts_tablet_manager.cc:1918] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:29.293540 21061 log.cc:1196] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/3e449ff0849f4c26a87b1e0c9513ecb6
I20260806 22:42:29.293874 21061 ts_tablet_manager.cc:1939] T 3e449ff0849f4c26a87b1e0c9513ecb6 P 02765bf5b21c4c18a8f260cc6329371a: Deleting consensus metadata
I20260806 22:42:29.294447 20230 catalog_manager.cc:4841] TS 02765bf5b21c4c18a8f260cc6329371a (127.0.186.195:35591): tablet 3e449ff0849f4c26a87b1e0c9513ecb6 (table test-table [id=a132256b9d1d4ed7b4a364aea4919121]) successfully deleted
W20260806 22:42:29.299546 21015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:29.298137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34949 (local address 127.0.0.1:60094)
0806 22:42:29.298196 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:29.298209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:29.298255 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:29.299494 (+ 1239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.194:34949: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:42:29.300199 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 20769
I20260806 22:42:29.327162 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 20201
I20260806 22:42:29.335397 747 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:42:29.669104 747 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:42:29.669193 747 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:42:30.013139 747 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:42:30.013 UTC [18810] LOG: received smart shutdown request
2026-08-06 22:42:30.017 UTC [18810] LOG: background worker "logical replication launcher" (PID 18823) exited with exit code 1
2026-08-06 22:42:30.017 UTC [18816] LOG: shutting down
2026-08-06 22:42:30.173 UTC [18810] LOG: database system is shut down
2026-08-06T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47517 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:30 dist-test-slave-gjrz krb5kdc[21075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:30 dist-test-slave-gjrz krb5kdc[21075](info): set up 2 sockets
Aug 06 22:42:30 dist-test-slave-gjrz krb5kdc[21075](info): commencing operation
krb5kdc: starting...
W20260806 22:42:31.394779 747 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:42:31.515465 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:34575
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:31.599426 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:31.599628 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:31.599648 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:31.600759 21094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:31.600795 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:31.600807 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:31.600818 21094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:31.600834 21094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:31.602321 21094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:34575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:31.602509 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:31.602686 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:31.607245 21105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.607251 21100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.607287 21101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:31.607403 21094 server_base.cc:1034] running on GCE node
I20260806 22:42:31.607730 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:31.607983 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:31.609125 21094 hybrid_clock.cc:648] HybridClock initialized: now 1786056151609109 us; error 28 us; skew 500 ppm
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:31.611446 21094 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:31.611841 21094 webserver.cc:460] Webserver started at http://127.0.186.254:34737/ using document root <none> and password file <none>
I20260806 22:42:31.612025 21094 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:31.612074 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:31.612197 21094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:31.613178 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "08361063a49a4df9a3bdfe9fac0a1e39"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "579891155e9d7dce6509653b2a8ccc9f"
server_key_iv: "1d3d9a804e5ee753c54f91646e6a6190"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.613502 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "08361063a49a4df9a3bdfe9fac0a1e39"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "579891155e9d7dce6509653b2a8ccc9f"
server_key_iv: "1d3d9a804e5ee753c54f91646e6a6190"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.614697 21094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:31.615357 21115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:31.615501 21094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:31.615592 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "08361063a49a4df9a3bdfe9fac0a1e39"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "579891155e9d7dce6509653b2a8ccc9f"
server_key_iv: "1d3d9a804e5ee753c54f91646e6a6190"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.615658 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:31.639662 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:31.641850 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:31.642011 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:31.645732 21094 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:34575
I20260806 22:42:31.645773 21196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:34575 every 8 connection(s)
I20260806 22:42:31.646086 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:31.646915 21199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:31.648550 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:31.652707 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21094
I20260806 22:42:31.652798 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:42:31.652985 747 external_mini_cluster.cc:1428] Setting key 7db2bb3f74b757e44f234f1100a6e6b5
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:31.657727 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.649194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34575 (local address 127.0.0.1:55684)
0806 22:42:31.649381 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.649424 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.649894 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.650163 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.650166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.650427 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.650574 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.650579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.651609 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.651610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.652267 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.652269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.652725 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.652726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.652801 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.653273 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.653281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.654681 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.656298 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.656307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.656309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.656557 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.656757 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.656758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.656758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.656882 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.657212 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.657219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.657281 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.657361 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.657583 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":81,"thread_start_us":83,"threads_started":1}
I20260806 22:42:31.657732 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.649246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.0.186.254:34575)
0806 22:42:31.649392 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.649401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.649511 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:42:31.650048 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.650050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.650066 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.650138 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.650666 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.651521 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.652342 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.652652 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.652745 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.654766 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.654774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.654774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.654781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.656218 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.656625 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.656627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.656628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.656699 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.656945 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.656946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.656946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.657142 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.657215 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.657393 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.657502 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260806 22:42:31.658468 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:31.658777 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.658681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34688)
0806 22:42:31.658721 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.658760 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:42:31.658895 21124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
I20260806 22:42:31.659255 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.659177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34698)
0806 22:42:31.659213 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.659237 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 22:42:31.659384 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:34495: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
I20260806 22:42:31.675980 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:34495
--webserver_interface=127.0.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:31.708583 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } attempt: 1
I20260806 22:42:31.709052 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.708936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34702)
0806 22:42:31.708989 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.709026 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:42:31.709395 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.709292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34710)
0806 22:42:31.709340 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.709368 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:42:31.709479 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:34495: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
W20260806 22:42:31.757498 21210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:31.757688 21210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:31.757717 21210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:31.758816 21210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:31.758854 21210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:31.758872 21210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:31.758888 21210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:31.758903 21210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:31.760407 21210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:34495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:31.760651 21210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:31.760859 21210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:31.765623 21226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.765599 21225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.765609 21228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:31.765728 21210 server_base.cc:1034] running on GCE node
I20260806 22:42:31.766191 21210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:31.766456 21210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:31.767597 21210 hybrid_clock.cc:648] HybridClock initialized: now 1786056151767584 us; error 27 us; skew 500 ppm
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:31.769924 21210 init.cc:377] Logged in from keytab as kudu/127.0.186.253@KRBTEST.COM (short username kudu)
I20260806 22:42:31.770330 21210 webserver.cc:460] Webserver started at http://127.0.186.253:38265/ using document root <none> and password file <none>
I20260806 22:42:31.770542 21210 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:31.770597 21210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:31.770722 21210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:31.771776 21210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data/instance:
uuid: "4784fbb947e446b0a6f055961adb445e"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "951f394d2a1f889e5b1a83c4bc73f37e"
server_key_iv: "8ac975d66ec0c596c4dd7df0e0582fe6"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.772107 21210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal/instance:
uuid: "4784fbb947e446b0a6f055961adb445e"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "951f394d2a1f889e5b1a83c4bc73f37e"
server_key_iv: "8ac975d66ec0c596c4dd7df0e0582fe6"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.773337 21210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:31.773998 21236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:31.774161 21210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:31.774242 21210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "4784fbb947e446b0a6f055961adb445e"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "951f394d2a1f889e5b1a83c4bc73f37e"
server_key_iv: "8ac975d66ec0c596c4dd7df0e0582fe6"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.774308 21210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:31.777604 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } attempt: 2
I20260806 22:42:31.777959 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.777837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34722)
0806 22:42:31.777889 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.777928 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:42:31.778231 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.778163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34738)
0806 22:42:31.778193 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.778216 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 22:42:31.778301 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:34495: Network error: Client connection negotiation failed: client connection to 127.0.186.253:34495: connect: Connection refused (error 111)
I20260806 22:42:31.786139 21210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:31.787840 21210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:31.787943 21210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:31.791884 21210 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.253:34495
I20260806 22:42:31.791926 21318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.253:34495 every 8 connection(s)
I20260806 22:42:31.792217 21210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
I20260806 22:42:31.793092 21320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:31.794593 21320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:31.800362 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21210
I20260806 22:42:31.800444 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-1/wal/instance
I20260806 22:42:31.800647 747 external_mini_cluster.cc:1428] Setting key bf3513670035a2b47130a9ee9659d954
I20260806 22:42:31.802961 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.795254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.0.186.254:34575)
0806 22:42:31.795323 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.795325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.795512 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:42:31.795971 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.795972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.795987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.796033 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.796636 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.797112 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.798044 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.798248 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.798303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.800299 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.800300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.800301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.800307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.801554 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.802012 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.802013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.802014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.802078 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.802326 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.802327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.802327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.802448 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.802509 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.802751 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.802839 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:42:31.803073 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.795212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34575 (local address 127.0.0.1:55700)
0806 22:42:31.795399 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.795435 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.795877 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.796110 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.796114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.796348 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.796548 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.796552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.797194 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.797195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.797961 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.797963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.798318 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.798319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.798443 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.798837 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.798845 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.800208 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.801632 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.801640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.801642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.801933 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.802154 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.802155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.802155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.802258 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.802507 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.802515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.802636 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.802720 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.802865 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260806 22:42:31.803539 21320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.186.252@KRBTEST.COM; defaulting to no policy
I20260806 22:42:31.810436 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.803758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34742)
0806 22:42:31.803812 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.803822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.803863 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.804121 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.804122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.804243 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.804286 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.804287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.805144 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.805145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.805632 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.805633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.806056 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.806057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.806116 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.806489 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.806493 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.807812 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.809385 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.809386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.809386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.809572 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.809744 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.809745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.809745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.809798 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.810040 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.810040 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.810077 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.810139 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.810276 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:42:31.810453 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.803795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.0.186.253:34495)
0806 22:42:31.804000 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.804003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.804020 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:42:31.804035 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.804037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.804047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.804107 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.804348 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.805067 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.805699 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.805994 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.806050 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.807889 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.807897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.807898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.807930 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.809311 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.809643 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.809645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.809646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.809691 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.809847 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.809848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.809849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.809979 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.810047 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.810176 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.810342 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.252@KRBTEST.COM" created.
I20260806 22:42:31.811124 21320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:31.811470 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.811343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47556)
0806 22:42:31.811387 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.811445 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:42:31.811636 21240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
I20260806 22:42:31.812036 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.811945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47572)
0806 22:42:31.811986 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.812015 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:42:31.812131 21320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:37699: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
I20260806 22:42:31.825544 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:37699
--webserver_interface=127.0.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:31.861346 21320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } attempt: 1
I20260806 22:42:31.861816 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.861678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47582)
0806 22:42:31.861736 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.861781 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:42:31.862159 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.862064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47596)
0806 22:42:31.862108 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.862137 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:42:31.862265 21320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:37699: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
I20260806 22:42:31.869427 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } attempt: 3
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:42:31.875648 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.869691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34744)
0806 22:42:31.869738 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.869749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.869797 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.869920 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.869921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.870077 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.870143 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.870145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.870794 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.870795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.871336 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.871339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.871703 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.871704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.871778 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.872171 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.872176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.873339 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.874721 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.874722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.874723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.874883 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.875047 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.875048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.875048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.875109 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.875346 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.875347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.875395 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.875453 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.875552 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:42:31.875648 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.869747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.0.186.253:34495)
0806 22:42:31.869801 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.869803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.869830 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:42:31.869856 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.869857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.869866 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.869906 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.870217 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.870720 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.871415 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.871641 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.871695 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.873416 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.873417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.873418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.873424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.874643 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.874942 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.874943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.874944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.874994 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.875159 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.875159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.875160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.875286 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.875343 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.875482 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.875560 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:42:31.876189 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:31.876515 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.876426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47612)
0806 22:42:31.876467 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.876496 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:42:31.876816 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.876736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47624)
0806 22:42:31.876769 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.876796 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 22:42:31.876892 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:37699: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
W20260806 22:42:31.906046 21330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:31.906242 21330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:31.906273 21330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:31.907434 21330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:31.907474 21330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:31.907493 21330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:31.907509 21330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:31.907526 21330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:31.908010 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } attempt: 1
I20260806 22:42:31.908396 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.908289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47628)
0806 22:42:31.908339 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.908369 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:42:31.908697 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.908619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47644)
0806 22:42:31.908654 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.908679 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 22:42:31.908795 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:37699: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
I20260806 22:42:31.909140 21330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:34575,127.0.186.253:34495,127.0.186.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:31.909377 21330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:31.909572 21330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:31.914037 21345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.914044 21342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:31.914041 21341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:31.914325 21330 server_base.cc:1034] running on GCE node
I20260806 22:42:31.914569 21330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:31.914858 21330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:31.915993 21330 hybrid_clock.cc:648] HybridClock initialized: now 1786056151915980 us; error 26 us; skew 500 ppm
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:31.918316 21330 init.cc:377] Logged in from keytab as kudu/127.0.186.252@KRBTEST.COM (short username kudu)
I20260806 22:42:31.918731 21330 webserver.cc:460] Webserver started at http://127.0.186.252:37423/ using document root <none> and password file <none>
I20260806 22:42:31.918949 21330 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:31.919009 21330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:31.919145 21330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:31.920190 21330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data/instance:
uuid: "8dd3589a91754d10b91efaf8840fa131"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "37c7ab46693d480235d81e96ff3ddb05"
server_key_iv: "190d073ce9bbbccd22b28c635ec6870b"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.920513 21330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal/instance:
uuid: "8dd3589a91754d10b91efaf8840fa131"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "37c7ab46693d480235d81e96ff3ddb05"
server_key_iv: "190d073ce9bbbccd22b28c635ec6870b"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.921701 21330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:31.922344 21357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:31.922495 21330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:31.922577 21330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "8dd3589a91754d10b91efaf8840fa131"
format_stamp: "Formatted at 2026-08-06 22:42:31 on dist-test-slave-gjrz"
server_key: "37c7ab46693d480235d81e96ff3ddb05"
server_key_iv: "190d073ce9bbbccd22b28c635ec6870b"
server_key_version: "encryptionkey@0"
I20260806 22:42:31.922643 21330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:31.930399 21320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } attempt: 2
I20260806 22:42:31.930742 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.930634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47650)
0806 22:42:31.930683 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.930716 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:42:31.931047 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.930969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47660)
0806 22:42:31.931003 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.931028 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 22:42:31.931126 21320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:37699: Network error: Client connection negotiation failed: client connection to 127.0.186.252:37699: connect: Connection refused (error 111)
I20260806 22:42:31.947045 21330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:31.949051 21330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:31.949191 21330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:31.953253 21330 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.252:37699
I20260806 22:42:31.953292 21443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.252:37699 every 8 connection(s)
I20260806 22:42:31.953595 21330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
I20260806 22:42:31.954438 21445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:31.955977 21445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:31.960036 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21330
I20260806 22:42:31.960142 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055997241161-747-0/minicluster-data/master-2/wal/instance
I20260806 22:42:31.960357 747 external_mini_cluster.cc:1428] Setting key 1ded816c431762281ff234bcd517f12f
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:31.964519 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.956588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.0.186.254:34575)
0806 22:42:31.956649 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.956651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.956815 (+ 164us) server_negotiation.cc:407] Connection header received
0806 22:42:31.957233 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.957234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.957244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.957287 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.957823 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.958284 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.959207 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.959395 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.959440 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.961730 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.961731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.961732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.961739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.963239 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.963633 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.963634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.963635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.963693 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.963907 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.963908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.963908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.964036 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.964102 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.964342 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.964432 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:42:31.964655 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.956563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34575 (local address 127.0.0.1:55706)
0806 22:42:31.956712 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.956745 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.957161 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.957346 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.957349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.957579 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.957761 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.957765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.958350 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.958350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.959147 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.959149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.959451 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.959452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.959552 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.959935 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.959945 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.961645 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.963332 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.963340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.963342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.963564 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.963750 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.963751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.963751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.963847 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.964093 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.964100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.964223 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.964307 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.964455 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260806 22:42:31.965174 21445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:42:31.968178 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.960711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.0.186.252:37699)
0806 22:42:31.960865 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.960869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.960910 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:42:31.960980 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.960981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.960997 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.961052 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.961759 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.962559 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.963280 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.963567 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.963633 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.965705 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.965713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.965713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.965720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.967021 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.967398 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.967402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.967402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.967474 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.967642 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.967643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.967643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.967764 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.967881 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.967995 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.968077 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:42:31.971483 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.965418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:34495 (local address 127.0.0.1:34750)
0806 22:42:31.965478 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.965489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.965531 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.965648 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.965649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.965765 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.965807 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.965808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.966640 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.966641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.967099 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.967101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.967503 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.967504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.967575 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.967910 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.967914 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.969139 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.970532 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.970533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.970534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.970732 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.970900 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.970901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.970901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.970957 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.971188 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.971189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.971228 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.971327 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.971414 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:42:31.971505 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.965482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.0.186.253:34495)
0806 22:42:31.965520 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.965521 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.965529 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:31.965592 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.965593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.965601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.965640 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.965877 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.966570 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.967176 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.967438 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.967505 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.969211 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.969212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.969212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.969218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.970459 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.970794 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.970794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.970795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.970838 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.971001 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.971002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.971002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.971134 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.971193 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.971350 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.971420 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:31.971822 21445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:42:31.975719 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.968847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.0.186.254:34575)
0806 22:42:31.968882 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.968884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.968892 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:31.968934 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.968935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.968944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.968991 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.969475 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.970123 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.970877 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.971128 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.971191 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.973367 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.973368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.973368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.973374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.974638 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.974923 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.974924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.974924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.975016 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.975174 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.975175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.975175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.975330 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.975514 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.975566 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.975640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:42:31.978063 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.972038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47674)
0806 22:42:31.972077 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.972086 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.972123 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.972232 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.972233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.972350 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.972372 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.972374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.972933 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.972934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.973545 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.973547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.973969 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.973970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.974007 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.974333 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.974338 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.975488 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.977142 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.977143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.977143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.977312 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.977481 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.977481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.977482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.977525 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.977756 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.977757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.977796 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.977850 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.977978 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:42:31.978145 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.972081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.0.186.252:37699)
0806 22:42:31.972111 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.972112 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.972119 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:31.972172 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.972173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.972181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.972228 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.972428 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.972870 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.973617 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.973898 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.973971 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.975569 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.975570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.975570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.975576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.977073 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.977361 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.977362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.977362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.977421 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.977570 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.977571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.977571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.977683 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.977756 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.977883 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.978044 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":17}
I20260806 22:42:31.979502 21445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: Bootstrap starting.
I20260806 22:42:31.980492 21445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: Neither blocks nor log segments found. Creating new log.
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:42:31.980770 21445 log.cc:824] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:31.981457 21445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: No bootstrap required, opened a new log
I20260806 22:42:31.982848 21445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:31.982965 21445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:31.982998 21445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd3589a91754d10b91efaf8840fa131, State: Initialized, Role: FOLLOWER
I20260806 22:42:31.983137 21445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:31.983457 21445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:31.983677 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.976621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.0.186.253:34495)
0806 22:42:31.976663 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.976664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.976671 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:31.976689 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.976690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.976698 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.976766 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.977200 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.977854 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.978631 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.978844 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.978892 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.981092 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.981093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.981093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.981099 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.982326 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.982616 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.982617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.982617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.982667 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.982824 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.982825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.982825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.982923 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.983501 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.983539 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.983620 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:31.983919 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } attempt: 2
I20260806 22:42:31.985095 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:31.985206 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:31.985281 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:31.985335 21470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:31.985375 21470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 22:42:31 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:42:31.986883 21330 master_runner.cc:186] Error getting master registration for 127.0.186.254:34575: OK, instance_id { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" instance_seqno: 1786056151644721 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:42:31.991005 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.984160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47688)
0806 22:42:31.984201 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:31.984209 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:31.984240 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:31.984376 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:31.984377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:31.984535 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:31.984580 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.984582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.985465 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.985466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.985932 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:31.985934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.986246 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.986247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:31.986298 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.986661 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:31.986666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:31.988075 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:31.989938 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.989939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.989939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.990143 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.990340 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:31.990340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:31.990341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:31.990396 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:31.990718 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:31.990719 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:31.990762 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:31.990820 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:31.990905 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:42:31.991381 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.984199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.0.186.252:37699)
0806 22:42:31.984235 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.984236 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.984245 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:31.984271 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.984272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.984280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.984654 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.984684 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.985387 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.985998 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.986191 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.986242 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.988164 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.988165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.988166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.988172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.989847 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.990221 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.990222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.990222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.990276 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.990475 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.990476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.990476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.990630 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.990691 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.991213 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.991306 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:31.991791 21330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:42:31.993194 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.986378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.0.186.254:34575)
0806 22:42:31.986416 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.986418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.986569 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:42:31.986657 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.986658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.986667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.986693 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.987437 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.988084 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.988988 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.989183 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.989235 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.990482 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.990483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.990483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.990489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.991944 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.992368 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.992369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.992369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.992424 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.992636 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.992637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.992637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.992755 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.992833 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.993045 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.993117 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:31.993247 21199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: Bootstrap starting.
I20260806 22:42:31.993393 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.986604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.0.186.253:34495)
0806 22:42:31.986647 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.986648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.986835 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:42:31.986932 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.986933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.986942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.986967 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.987669 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.988293 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.989255 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.989521 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.989561 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.990837 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.990839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.990839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.990845 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.992236 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.992606 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.992607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.992608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.992669 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.992867 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.992867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.992868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.992982 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:31.993043 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:31.993238 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:42:31.993316 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:31.994072 21199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:31.994400 21199 log.cc:824] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:31.995309 21199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: No bootstrap required, opened a new log
I20260806 22:42:31.997138 21199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:31.997277 21199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:31.997311 21199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08361063a49a4df9a3bdfe9fac0a1e39, State: Initialized, Role: FOLLOWER
I20260806 22:42:31.997484 21199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:31.997814 21199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:31.997782 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:31.999356 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:32.000008 21500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:32.000061 21500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:32.000883 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:31.992038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.0.186.252:37699)
0806 22:42:31.992071 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:31.992072 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:31.993869 (+ 1797us) server_negotiation.cc:407] Connection header received
0806 22:42:31.993890 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:31.993891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:31.993900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:31.993940 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:31.994699 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.995379 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.996259 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:31.996550 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:31.996620 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:31.997856 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:31.997857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:31.997858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:31.997865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:31.999445 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:31.999815 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:31.999816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:31.999816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:31.999873 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.000120 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.000121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.000121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.000253 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.000344 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.000739 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.000807 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:32.000710 21094 master_runner.cc:186] Error getting master registration for 127.0.186.253:34495: OK, instance_id { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" instance_seqno: 1786056151790691 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:42:32.002810 21094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:42:32.022257 21320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } attempt: 3
Aug 06 22:42:32 dist-test-slave-gjrz krb5kdc[21075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:42:32.028658 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.022589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.0.186.252:37699)
0806 22:42:32.022627 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:32.022629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:32.022637 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:32.022692 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:32.022692 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:32.022702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:32.022745 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:32.023013 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.023616 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.024316 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.024563 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.024620 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.026414 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:32.026415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:32.026415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:32.026422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:32.027691 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.027983 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.027984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.027984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.028034 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:32.028201 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:32.028202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:32.028202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:32.028300 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:32.028354 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:32.028499 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:42:32.028567 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:42:32.028654 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:32.022517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:37699 (local address 127.0.0.1:47700)
0806 22:42:32.022574 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:32.022586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:32.022630 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:32.022749 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:32.022749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:32.022904 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:32.022950 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.022951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.023698 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.023699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.024244 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:32.024247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:32.024631 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:32.024632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:32.024688 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:32.025100 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:32.025106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:32.026327 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:32.027763 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.027765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.027765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.027924 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.028094 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:32.028094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:32.028095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:32.028153 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:32.028354 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:32.028355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:32.028398 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:42:32.028457 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:32.028553 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:42:32.030160 21320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: Bootstrap starting.
I20260806 22:42:32.030782 21320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:32.031059 21320 log.cc:824] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:32.031790 21320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: No bootstrap required, opened a new log
I20260806 22:42:32.033232 21320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:32.033341 21320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:32.033373 21320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4784fbb947e446b0a6f055961adb445e, State: Initialized, Role: FOLLOWER
I20260806 22:42:32.033509 21320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:32.033818 21320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:42:32.035696 21522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:32.035799 21522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:32.036103 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.036206 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:32.037671 21210 master_runner.cc:263] No leader master found from master 4784fbb947e446b0a6f055961adb445e
I20260806 22:42:32.037714 21210 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:42:32.038456 21252 tablet.cc:2369] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:32.108330 21505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:32.108597 21505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:32.109174 21163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4784fbb947e446b0a6f055961adb445e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" is_pre_election: true
I20260806 22:42:32.109324 21163 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4784fbb947e446b0a6f055961adb445e in term 0.
I20260806 22:42:32.109637 21242 leader_election.cc:302] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08361063a49a4df9a3bdfe9fac0a1e39, 4784fbb947e446b0a6f055961adb445e; no voters:
I20260806 22:42:32.110164 21406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4784fbb947e446b0a6f055961adb445e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd3589a91754d10b91efaf8840fa131" is_pre_election: true
I20260806 22:42:32.110312 21406 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4784fbb947e446b0a6f055961adb445e in term 0.
I20260806 22:42:32.110569 21505 leader_election.cc:288] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08361063a49a4df9a3bdfe9fac0a1e39 (127.0.186.254:34575), 8dd3589a91754d10b91efaf8840fa131 (127.0.186.252:37699)
I20260806 22:42:32.110709 21505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:32.110805 21505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:32.110880 21505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:32.111933 21505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:32.112250 21163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4784fbb947e446b0a6f055961adb445e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08361063a49a4df9a3bdfe9fac0a1e39"
I20260806 22:42:32.112335 21163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:32.113294 21163 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4784fbb947e446b0a6f055961adb445e in term 1.
I20260806 22:42:32.113529 21242 leader_election.cc:302] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08361063a49a4df9a3bdfe9fac0a1e39, 4784fbb947e446b0a6f055961adb445e; no voters:
I20260806 22:42:32.113816 21406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4784fbb947e446b0a6f055961adb445e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd3589a91754d10b91efaf8840fa131"
I20260806 22:42:32.113900 21406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:32.114807 21406 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4784fbb947e446b0a6f055961adb445e in term 1.
I20260806 22:42:32.115046 21505 leader_election.cc:288] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [CANDIDATE]: Term 1 election: Requested vote from peers 08361063a49a4df9a3bdfe9fac0a1e39 (127.0.186.254:34575), 8dd3589a91754d10b91efaf8840fa131 (127.0.186.252:37699)
I20260806 22:42:32.115173 21505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:32.115253 21505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [term 1 LEADER]: Becoming Leader. State: Replica: 4784fbb947e446b0a6f055961adb445e, State: Running, Role: LEADER
I20260806 22:42:32.115401 21505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } }
I20260806 22:42:32.115926 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4784fbb947e446b0a6f055961adb445e. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.116065 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:32.116330 21530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:32.116566 21530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:32.118685 21406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [term 1 FOLLOWER]: Refusing update from remote peer 4784fbb947e446b0a6f055961adb445e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:32.119087 21527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:32.120554 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4784fbb947e446b0a6f055961adb445e. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.120662 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:32.121088 21163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [term 1 FOLLOWER]: Refusing update from remote peer 4784fbb947e446b0a6f055961adb445e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:32.121389 21527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [LEADER]: Connected to new peer: Peer: permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:32.123551 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4784fbb947e446b0a6f055961adb445e. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.123657 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:32.124383 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.124490 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:32.125083 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.125182 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:32.125321 21531 mvcc.cc:205] Tried to move back new op lower bound from 7315685999075573760 to 7315685999065636864. Current Snapshot: MvccSnapshot[applied={T|T < 7315685999075573760}]
I20260806 22:42:32.126034 21536 mvcc.cc:205] Tried to move back new op lower bound from 7315685999075573760 to 7315685999065636864. Current Snapshot: MvccSnapshot[applied={T|T < 7315685999075573760}]
I20260806 22:42:32.126803 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.126879 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:32.127249 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4784fbb947e446b0a6f055961adb445e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08361063a49a4df9a3bdfe9fac0a1e39" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34575 } } peers { permanent_uuid: "4784fbb947e446b0a6f055961adb445e" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 34495 } } peers { permanent_uuid: "8dd3589a91754d10b91efaf8840fa131" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 37699 } } }
I20260806 22:42:32.127344 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:32.128139 21530 catalog_manager.cc:1347] Generated new cluster ID: 7506b1ff99a941538aed747489aee7c2
I20260806 22:42:32.128183 21530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:32.136003 21532 mvcc.cc:205] Tried to move back new op lower bound from 7315685999075573760 to 7315685999065636864. Current Snapshot: MvccSnapshot[applied={T|T < 7315685999075573760}]
I20260806 22:42:32.136169 21530 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:32.136891 21530 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:32.144784 21530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4784fbb947e446b0a6f055961adb445e: Generated new TSK 0
I20260806 22:42:32.144991 21530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:32.985790 21470 catalog_manager.cc:1259] Loaded cluster ID: 7506b1ff99a941538aed747489aee7c2
I20260806 22:42:32.985862 21470 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: loading cluster ID for follower catalog manager: success
I20260806 22:42:32.986691 21470 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: acquiring CA information for follower catalog manager: success
I20260806 22:42:32.986984 21470 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8dd3589a91754d10b91efaf8840fa131: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:33.000451 21500 catalog_manager.cc:1259] Loaded cluster ID: 7506b1ff99a941538aed747489aee7c2
I20260806 22:42:33.000512 21500 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: loading cluster ID for follower catalog manager: success
I20260806 22:42:33.001304 21500 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: acquiring CA information for follower catalog manager: success
I20260806 22:42:33.001603 21500 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08361063a49a4df9a3bdfe9fac0a1e39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:33.204638 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21094
I20260806 22:42:33.215144 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21210
I20260806 22:42:33.225786 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21330
2026-08-06T22:42:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2971 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:33 dist-test-slave-gjrz krb5kdc[21560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:33 dist-test-slave-gjrz krb5kdc[21560](info): set up 2 sockets
Aug 06 22:42:33 dist-test-slave-gjrz krb5kdc[21560](info): commencing operation
krb5kdc: starting...
W20260806 22:42:34.355916 747 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:42:34.465619 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:32831
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:32831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:34.544075 21577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:34.544268 21577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:34.544298 21577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:34.545406 21577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:34.545446 21577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:34.545464 21577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:34.545481 21577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:34.545496 21577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:34.546945 21577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:32831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:32831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:34.547181 21577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:34.547382 21577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:34.551846 21584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:34.551868 21586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.552030 21577 server_base.cc:1034] running on GCE node
W20260806 22:42:34.551872 21588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:34.552361 21577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:34.552603 21577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:34.553771 21577 hybrid_clock.cc:648] HybridClock initialized: now 1786056154553751 us; error 28 us; skew 500 ppm
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:34.556155 21577 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:34.556598 21577 webserver.cc:460] Webserver started at http://127.0.186.254:41599/ using document root <none> and password file <none>
I20260806 22:42:34.556803 21577 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:34.556849 21577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:34.556973 21577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:34.557971 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "9a9b48b2290c44fe8a2cd17419c3dda5"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-gjrz"
server_key: "5cad6095f5bafc77f81c1ad0f36359fb"
server_key_iv: "ec4b59b6f42794fc6e8b0846a3e914a5"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.558300 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "9a9b48b2290c44fe8a2cd17419c3dda5"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-gjrz"
server_key: "5cad6095f5bafc77f81c1ad0f36359fb"
server_key_iv: "ec4b59b6f42794fc6e8b0846a3e914a5"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.559661 21577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:34.560371 21603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:34.560531 21577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:34.560624 21577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "9a9b48b2290c44fe8a2cd17419c3dda5"
format_stamp: "Formatted at 2026-08-06 22:42:34 on dist-test-slave-gjrz"
server_key: "5cad6095f5bafc77f81c1ad0f36359fb"
server_key_iv: "ec4b59b6f42794fc6e8b0846a3e914a5"
server_key_version: "encryptionkey@0"
I20260806 22:42:34.560685 21577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:34.570567 21577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:34.572395 21577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:34.572515 21577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:34.576295 21577 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:32831
I20260806 22:42:34.576336 21692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:32831 every 8 connection(s)
I20260806 22:42:34.576650 21577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:34.577502 21695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:34.579751 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21577
I20260806 22:42:34.579807 21695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5: Bootstrap starting.
I20260806 22:42:34.579841 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:42:34.580020 747 external_mini_cluster.cc:1428] Setting key 76874abfdf90d65dd23630fad94973d1
I20260806 22:42:34.580466 21695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:34.580843 21695 log.cc:824] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:34.581581 21695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5: No bootstrap required, opened a new log
I20260806 22:42:34.583920 21695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } }
I20260806 22:42:34.584059 21695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:34.584080 21695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9b48b2290c44fe8a2cd17419c3dda5, State: Initialized, Role: FOLLOWER
I20260806 22:42:34.584193 21695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } }
I20260806 22:42:34.584260 21695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:34.584285 21695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:34.584365 21695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:34.585054 21695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } }
I20260806 22:42:34.585166 21695 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9b48b2290c44fe8a2cd17419c3dda5; no voters:
I20260806 22:42:34.585317 21695 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:34.585470 21705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:34.585575 21695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:34.585649 21705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [term 1 LEADER]: Becoming Leader. State: Replica: 9a9b48b2290c44fe8a2cd17419c3dda5, State: Running, Role: LEADER
I20260806 22:42:34.585736 21705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } }
I20260806 22:42:34.586105 21705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9b48b2290c44fe8a2cd17419c3dda5. Latest consensus state: current_term: 1 leader_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } } }
I20260806 22:42:34.586170 21703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9b48b2290c44fe8a2cd17419c3dda5" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 32831 } } }
I20260806 22:42:34.586246 21705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.586256 21703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:34.586474 21713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:34.587013 21713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:34.588965 21713 catalog_manager.cc:1347] Generated new cluster ID: d3a45a856b694308a30134eefb967fd5
I20260806 22:42:34.589027 21713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:34.590770 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:34.580467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.0.186.254:32831)
0806 22:42:34.580677 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:34.580688 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:34.580726 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:42:34.581278 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:34.581288 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:34.581352 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:34.581499 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:34.582116 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.583542 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.584329 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:34.584912 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:34.585041 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:34.587588 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:34.587592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:34.587593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:34.587603 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:34.589275 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.589579 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.589582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.589583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.589670 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:34.589845 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:34.589845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:34.589846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:34.590067 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:34.590347 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:34.590463 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:42:34.590599 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260806 22:42:34.591647 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:34.595338 21713 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:34.595806 21713 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:34.603695 21713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9b48b2290c44fe8a2cd17419c3dda5: Generated new TSK 0
I20260806 22:42:34.603855 21713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:42:34.645184 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21577
2026-08-06T22:42:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1419 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21735](info): set up 2 sockets
Aug 06 22:42:34 dist-test-slave-gjrz krb5kdc[21735](info): commencing operation
krb5kdc: starting...
W20260806 22:42:35.772468 747 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:42:35 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:42:35.886462 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:35.967021 21757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:35.967218 21757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:35.967239 21757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:35.968366 21757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:35.968406 21757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:35.968420 21757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:35.968430 21757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:35.968441 21757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:35.969905 21757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:35.970127 21757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:35.970328 21757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:35.974810 21768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.974802 21765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:35.974802 21770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:35.974916 21757 server_base.cc:1034] running on GCE node
I20260806 22:42:35.975238 21757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:35.975519 21757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:35.976649 21757 hybrid_clock.cc:648] HybridClock initialized: now 1786056155976641 us; error 26 us; skew 500 ppm
Aug 06 22:42:35 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:35.978914 21757 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:35.979357 21757 webserver.cc:460] Webserver started at http://127.0.186.254:34255/ using document root <none> and password file <none>
I20260806 22:42:35.979545 21757 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:35.979588 21757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:35.979705 21757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:35.980732 21757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "025c514b9ce84c92854eb4c2a4a30ea8"
format_stamp: "Formatted at 2026-08-06 22:42:35 on dist-test-slave-gjrz"
server_key: "594623052d20fc2a88bbee53cf4d4084"
server_key_iv: "d9c478e6a6d2c7c535d29eb986d7a3e0"
server_key_version: "encryptionkey@0"
I20260806 22:42:35.981057 21757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "025c514b9ce84c92854eb4c2a4a30ea8"
format_stamp: "Formatted at 2026-08-06 22:42:35 on dist-test-slave-gjrz"
server_key: "594623052d20fc2a88bbee53cf4d4084"
server_key_iv: "d9c478e6a6d2c7c535d29eb986d7a3e0"
server_key_version: "encryptionkey@0"
I20260806 22:42:35.982260 21757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:35.982862 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:35.983012 21757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:35.983098 21757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "025c514b9ce84c92854eb4c2a4a30ea8"
format_stamp: "Formatted at 2026-08-06 22:42:35 on dist-test-slave-gjrz"
server_key: "594623052d20fc2a88bbee53cf4d4084"
server_key_iv: "d9c478e6a6d2c7c535d29eb986d7a3e0"
server_key_version: "encryptionkey@0"
I20260806 22:42:35.983152 21757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:35.999907 21757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:36.001623 21757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:36.001726 21757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:36.005407 21757 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:38545
I20260806 22:42:36.005442 21869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:38545 every 8 connection(s)
I20260806 22:42:36.005738 21757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:36.006665 21870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:36.008895 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap starting.
I20260806 22:42:36.009465 21870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:36.009727 21870 log.cc:824] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:36.010372 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: No bootstrap required, opened a new log
I20260806 22:42:36.010785 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21757
I20260806 22:42:36.010903 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:42:36.011133 747 external_mini_cluster.cc:1428] Setting key 736c092f070ad600a291c479e5676aae
I20260806 22:42:36.012404 21870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.012554 21870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:36.012636 21870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Initialized, Role: FOLLOWER
I20260806 22:42:36.012769 21870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.012852 21870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:36.012892 21870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:36.012939 21870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:36.013831 21870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.013921 21870 leader_election.cc:302] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 025c514b9ce84c92854eb4c2a4a30ea8; no voters:
I20260806 22:42:36.014065 21870 leader_election.cc:288] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:36.014106 21881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:36.014226 21881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 LEADER]: Becoming Leader. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Running, Role: LEADER
I20260806 22:42:36.014322 21870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:36.014423 21881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.014793 21881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025c514b9ce84c92854eb4c2a4a30ea8. Latest consensus state: current_term: 1 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:36.015012 21881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:36.014950 21883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:36.015339 21883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:36.015542 21894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:36.016101 21894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:36.017409 21894 catalog_manager.cc:1347] Generated new cluster ID: 69e999ced576448992e259226d8b2133
I20260806 22:42:36.017459 21894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:36.022318 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.011665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.0.186.254:38545)
0806 22:42:36.011838 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.011845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.011871 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:42:36.012370 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.012375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.012417 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.012525 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:36.013287 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.014327 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.015314 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.015903 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.016006 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.018682 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:36.018686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:36.018686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:36.018697 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:36.020595 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.020992 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.020995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.020995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.021119 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.021312 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.021313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.021313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.021581 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:36.021724 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.021831 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.021973 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260806 22:42:36.024521 21894 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:36.025214 21894 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:36.040145 21894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Generated new TSK 0
I20260806 22:42:36.040328 21894 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:42:36.069397 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:36.151681 21914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:36.151870 21914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:36.151890 21914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:36.153005 21914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:36.153051 21914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:36.153076 21914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:36.154543 21914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:36.154759 21914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:36.154985 21914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:36.159787 21920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.159849 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.160002 21914 server_base.cc:1034] running on GCE node
W20260806 22:42:36.159787 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.160243 21914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:36.160496 21914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:36.161644 21914 hybrid_clock.cc:648] HybridClock initialized: now 1786056156161633 us; error 29 us; skew 500 ppm
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:36.164157 21914 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:36.164546 21914 webserver.cc:460] Webserver started at http://127.0.186.193:38627/ using document root <none> and password file <none>
I20260806 22:42:36.164747 21914 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:36.164788 21914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:36.164880 21914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:36.165861 21914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "7871df1715394f92a633b438c5b258de"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "7ea057829a4813544463131ec6b3816b"
server_key_iv: "a5b0523f7dd26ae64972f4d7233fe60f"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.166167 21914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "7871df1715394f92a633b438c5b258de"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "7ea057829a4813544463131ec6b3816b"
server_key_iv: "a5b0523f7dd26ae64972f4d7233fe60f"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.167361 21914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:36.168011 21951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.168172 21914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:36.168258 21914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "7871df1715394f92a633b438c5b258de"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "7ea057829a4813544463131ec6b3816b"
server_key_iv: "a5b0523f7dd26ae64972f4d7233fe60f"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.168324 21914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:36.186877 21914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:36.188675 21914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:36.188800 21914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:36.189023 21914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:36.189316 21914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:36.189348 21914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.189370 21914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:36.189394 21914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.194698 21914 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:36915
I20260806 22:42:36.194770 22133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:36915 every 8 connection(s)
I20260806 22:42:36.195037 21914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:36.203495 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.195743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:46767 (local address 127.0.186.254:38545)
0806 22:42:36.195805 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.195808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.195978 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:42:36.196484 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.196485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.196502 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.196553 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:36.197467 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.197967 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.198920 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.199143 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.199199 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.201193 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:36.201194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:36.201195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:36.201202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:36.202441 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.202762 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.202762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.202763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.202809 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.202974 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.202975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.202975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.203085 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:36.203132 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.203364 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.203431 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:42:36.203666 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.195698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:46767)
0806 22:42:36.195867 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.195905 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.196386 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.196630 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.196634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.197179 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:36.197371 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.197377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.198052 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.198054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.198838 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.198841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.199217 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.199218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.199367 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.199794 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:36.199801 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:36.201099 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:36.202504 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.202506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.202508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.202711 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.202852 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.202853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.202853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.202935 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.203128 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:36.203131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:36.203237 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.203347 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.203463 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260806 22:42:36.204111 22135 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:36.204191 22135 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:36.204355 22135 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:36.204838 21809 ts_manager.cc:194] Registered new tserver with Master: 7871df1715394f92a633b438c5b258de (127.0.186.193:36915)
I20260806 22:42:36.205132 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 21914
I20260806 22:42:36.205226 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:36.205402 747 external_mini_cluster.cc:1428] Setting key 548a7da8b062397e6e493934ec99ab41
I20260806 22:42:36.205498 21809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:46767
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:42:36.227674 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:36.308341 22140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:36.308547 22140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:36.308588 22140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:36.309738 22140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:36.309778 22140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:36.309813 22140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:36.311250 22140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:36.311512 22140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:36.311723 22140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:36.316493 22148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.316601 22151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.316512 22146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.316725 22140 server_base.cc:1034] running on GCE node
I20260806 22:42:36.316983 22140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:36.317211 22140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:36.318370 22140 hybrid_clock.cc:648] HybridClock initialized: now 1786056156318355 us; error 30 us; skew 500 ppm
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:36.320755 22140 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:36.321175 22140 webserver.cc:460] Webserver started at http://127.0.186.194:39777/ using document root <none> and password file <none>
I20260806 22:42:36.321376 22140 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:36.321424 22140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:36.321547 22140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:36.322602 22140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "2e41f688ed4d4d768246208cd34e2a49"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5489c894468f538c1426443e7750c631"
server_key_iv: "5e607f0a314a5da3ad62e4ad0bbf5ea2"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.322937 22140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "2e41f688ed4d4d768246208cd34e2a49"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5489c894468f538c1426443e7750c631"
server_key_iv: "5e607f0a314a5da3ad62e4ad0bbf5ea2"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.324200 22140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:42:36.324890 22160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.325067 22140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.325166 22140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "2e41f688ed4d4d768246208cd34e2a49"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5489c894468f538c1426443e7750c631"
server_key_iv: "5e607f0a314a5da3ad62e4ad0bbf5ea2"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.325227 22140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:36.332158 22140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:36.333969 22140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:36.334103 22140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:36.334296 22140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:36.334574 22140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:36.334617 22140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.334645 22140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:36.334667 22140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.340719 22140 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34643
I20260806 22:42:36.340780 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34643 every 8 connection(s)
I20260806 22:42:36.341068 22140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:36.342312 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 22140
I20260806 22:42:36.342418 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:36.342624 747 external_mini_cluster.cc:1428] Setting key 7ea3e2be6ca579a63e0c6e145d7aec1b
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:36.350983 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.341827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:51113 (local address 127.0.186.254:38545)
0806 22:42:36.341888 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.341890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.342066 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:42:36.342757 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.342759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.342771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.342852 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:36.344051 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.344543 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.345779 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.346070 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.346153 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.348414 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:36.348416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:36.348416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:36.348424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:36.349690 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.350075 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.350076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.350077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.350131 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.350355 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.350356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.350357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.350471 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:36.350531 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.350798 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.350891 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:42:36.351198 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.341788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:51113)
0806 22:42:36.341949 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.341991 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.342615 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.342918 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.342923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.343674 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:36.343937 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.343944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.344645 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.344646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.345682 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.345685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.346153 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.346154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.346253 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.346736 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:36.346746 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:36.348320 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:36.349769 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.349777 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.349779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.350006 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.350189 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.350189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.350190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.350299 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.350535 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:36.350542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:36.350681 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.350770 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.350931 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 22:42:36.351687 22354 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:36.351794 22354 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
I20260806 22:42:36.352000 22354 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:36.352375 21808 ts_manager.cc:194] Registered new tserver with Master: 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
I20260806 22:42:36.352759 21808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:51113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:42:36.368242 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:36.450024 22359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:36.450218 22359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:36.450240 22359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:36.451354 22359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:36.451386 22359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:36.451411 22359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:36.452816 22359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:36.453022 22359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:36.453226 22359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:36.457964 22370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.457960 22366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.457983 22367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.458175 22359 server_base.cc:1034] running on GCE node
I20260806 22:42:36.458351 22359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:36.458587 22359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:36.459738 22359 hybrid_clock.cc:648] HybridClock initialized: now 1786056156459719 us; error 28 us; skew 500 ppm
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:36.462193 22359 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:36.462611 22359 webserver.cc:460] Webserver started at http://127.0.186.195:45673/ using document root <none> and password file <none>
I20260806 22:42:36.462823 22359 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:36.462872 22359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:36.463009 22359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:36.464046 22359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5e2b890a3bd9e257539c951e241c7cfa"
server_key_iv: "96e1cc33232acffe68bc0957eb52a3e8"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.464376 22359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5e2b890a3bd9e257539c951e241c7cfa"
server_key_iv: "96e1cc33232acffe68bc0957eb52a3e8"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.465627 22359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:36.466356 22382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.466524 22359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:36.466612 22359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5e2b890a3bd9e257539c951e241c7cfa"
server_key_iv: "96e1cc33232acffe68bc0957eb52a3e8"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.466672 22359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:36.496619 22359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:36.498477 22359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:36.498618 22359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:36.498834 22359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:36.499126 22359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:36.499159 22359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.499187 22359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:36.499210 22359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.505211 22359 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35205
I20260806 22:42:36.505270 22574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35205 every 8 connection(s)
I20260806 22:42:36.505570 22359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:36.513814 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 22359
I20260806 22:42:36.513944 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:36.514176 747 external_mini_cluster.cc:1428] Setting key 7401a32011f3c87d79b6bf340e3656d0
I20260806 22:42:36.514297 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.506265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:53867 (local address 127.0.186.254:38545)
0806 22:42:36.506326 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.506328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.506520 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:42:36.507020 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.507021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.507030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.507067 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:36.507981 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.508466 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.509396 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.509567 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.509619 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.511699 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:36.511701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:36.511701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:36.511708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:36.512971 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.513362 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.513363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.513363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.513420 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.513609 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.513609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.513609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.513715 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:36.513762 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.514121 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.514202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:42:36.514468 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.506229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:53867)
0806 22:42:36.506401 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.506443 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.506927 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.507135 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.507138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.507717 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:36.507914 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.507919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.508531 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.508531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.509336 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.509338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.509630 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.509631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.509733 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.510214 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:36.510223 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:36.511588 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:36.513066 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.513072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.513074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.513295 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.513474 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:36.513474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:36.513475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:36.513565 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:36.513770 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:36.513777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:36.513954 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.514073 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.514241 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260806 22:42:36.515034 22575 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:36.515138 22575 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:36.515347 22575 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:36.515761 21809 ts_manager.cc:194] Registered new tserver with Master: 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205)
I20260806 22:42:36.516148 21809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:53867
I20260806 22:42:36.525830 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:36.532238 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.527032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.0.186.254:38545)
0806 22:42:36.527075 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.527077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.527190 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:42:36.527237 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.527238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.527245 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.527317 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:36.527856 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.528332 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.529058 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.529242 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.529290 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.530326 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:36.530326 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:36.530327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:36.530331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:36.531406 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.531692 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.531692 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.531693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.531737 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:36.531855 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:36.531855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:36.531856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:36.531978 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:36.532045 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.532125 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.532180 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:36.533547 21809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:36.535043 21809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:36.540971 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.536950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35205 (local address 127.0.0.1:48708)
0806 22:42:36.537263 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.537274 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.537335 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.537559 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.537560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.537562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:36.537613 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.537615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.539010 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.539011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.539885 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.539889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.540690 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.540691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.540758 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.540768 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.540827 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.540890 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260806 22:42:36.540974 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.537072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.0.186.195:35205)
0806 22:42:36.537227 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.537232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.537388 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:42:36.537426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.537428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.537442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.537500 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:36.537758 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.538873 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.539954 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.540535 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.540596 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.540655 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.540856 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.540900 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260806 22:42:36.541006 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.537024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.0.186.194:34643)
0806 22:42:36.537305 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.537309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.537318 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:36.537356 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.537359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.537375 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.537445 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:36.537679 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.538550 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.539910 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.540566 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.540632 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.540685 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.540878 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.540923 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":118,"threads_started":1}
I20260806 22:42:36.541375 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.537370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.0.186.193:36915)
0806 22:42:36.537573 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.537578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.537587 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:36.537633 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.537635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.537650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.537722 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:36.537802 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.538873 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.540207 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.540955 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.541013 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.541065 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.541252 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.541298 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260806 22:42:36.541200 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.536949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34643 (local address 127.0.0.1:33900)
0806 22:42:36.537135 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.537176 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.537238 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.537451 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.537453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.537455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:36.537521 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.537522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.538648 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.538649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.539824 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.539827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.540721 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.540722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.540782 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.540786 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.540832 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.541134 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":249,"thread_start_us":64,"threads_started":1}
I20260806 22:42:36.541293 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.537168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:36915 (local address 127.0.0.1:48602)
0806 22:42:36.537335 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.537343 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.537377 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.537727 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.537728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.537729 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:36.537760 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.537762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.538979 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.538980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.540122 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.540125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.541102 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.541104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.541159 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.541162 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.541218 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.541252 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":58,"threads_started":1}
I20260806 22:42:36.542096 22471 tablet_service.cc:1467] Processing CreateTablet for tablet bf425c270c1243ffa52d1e319013a224 (DEFAULT_TABLE table=test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:36.542105 22247 tablet_service.cc:1467] Processing CreateTablet for tablet bf425c270c1243ffa52d1e319013a224 (DEFAULT_TABLE table=test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:36.542346 22037 tablet_service.cc:1467] Processing CreateTablet for tablet bf425c270c1243ffa52d1e319013a224 (DEFAULT_TABLE table=test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:36.542445 22247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf425c270c1243ffa52d1e319013a224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:36.542445 22471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf425c270c1243ffa52d1e319013a224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:36.542609 22037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf425c270c1243ffa52d1e319013a224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:36.545193 22596 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Bootstrap starting.
I20260806 22:42:36.545611 22597 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Bootstrap starting.
I20260806 22:42:36.545801 22598 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Bootstrap starting.
I20260806 22:42:36.545848 22596 tablet_bootstrap.cc:654] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:36.546248 22597 tablet_bootstrap.cc:654] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:36.546394 22598 tablet_bootstrap.cc:654] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:36.546465 22596 log.cc:824] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:36.546819 22597 log.cc:824] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:36.546922 22598 log.cc:824] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:36.547291 22596 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: No bootstrap required, opened a new log
I20260806 22:42:36.547364 22596 ts_tablet_manager.cc:1397] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:36.547696 22598 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: No bootstrap required, opened a new log
I20260806 22:42:36.547756 22598 ts_tablet_manager.cc:1397] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:36.547849 22597 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: No bootstrap required, opened a new log
I20260806 22:42:36.547920 22597 ts_tablet_manager.cc:1397] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:36.549180 22598 raft_consensus.cc:348] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.549324 22598 raft_consensus.cc:374] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:36.549357 22598 raft_consensus.cc:729] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e41f688ed4d4d768246208cd34e2a49, State: Initialized, Role: FOLLOWER
I20260806 22:42:36.549273 22596 raft_consensus.cc:348] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.549479 22596 raft_consensus.cc:374] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:36.549492 22598 consensus_queue.cc:260] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.549535 22596 raft_consensus.cc:729] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7871df1715394f92a633b438c5b258de, State: Initialized, Role: FOLLOWER
I20260806 22:42:36.549674 22596 consensus_queue.cc:260] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.549782 22354 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
I20260806 22:42:36.549789 22597 raft_consensus.cc:348] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.549954 22597 raft_consensus.cc:374] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:36.549991 22597 raft_consensus.cc:729] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e86bf4957fb45abb1ffe19e460f23e4, State: Initialized, Role: FOLLOWER
I20260806 22:42:36.550001 22596 ts_tablet_manager.cc:1428] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:36.550103 22135 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
I20260806 22:42:36.550020 22598 ts_tablet_manager.cc:1428] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:36.550101 22597 consensus_queue.cc:260] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.550375 22597 ts_tablet_manager.cc:1428] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:42:36.550462 22575 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
W20260806 22:42:36.591647 22355 tablet.cc:2369] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:36.695760 22136 tablet.cc:2369] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:36.737851 22614 raft_consensus.cc:482] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:36.737970 22614 raft_consensus.cc:504] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.738296 22614 leader_election.cc:288] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643), 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205)
I20260806 22:42:36.741192 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.738548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:39085 (local address 127.0.186.195:35205)
0806 22:42:36.738592 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.738594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.738680 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:42:36.738748 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.738749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.738759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.738804 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:36.738972 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.739482 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740487 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.740869 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740932 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.740942 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.741093 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.741130 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:36.741205 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.738524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:56227 (local address 127.0.186.194:34643)
0806 22:42:36.738584 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.738586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.738691 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:42:36.738756 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.738757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.738767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.738809 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:36.738983 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.739496 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740492 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.740877 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740939 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.740947 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.741095 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.741136 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:42:36.741225 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.738472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34643 (local address 127.0.186.193:56227)
0806 22:42:36.738627 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.738636 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.738695 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.738821 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.738822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.738824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:36.738926 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.738927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.739562 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.739563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.740403 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.740406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.740948 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.741028 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.741031 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.741084 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.741130 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":20,"thread_start_us":78,"threads_started":1}
I20260806 22:42:36.741242 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.738472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35205 (local address 127.0.186.193:39085)
0806 22:42:36.738614 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.738624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:36.738683 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:36.738816 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:36.738817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:36.738819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:36.738913 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.738915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.739553 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.739554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.740399 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:36.740401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.740945 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.740946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:36.741027 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.741030 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:36.741076 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:42:36.741118 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":48,"thread_start_us":67,"threads_started":1}
I20260806 22:42:36.741523 22280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf425c270c1243ffa52d1e319013a224" candidate_uuid: "7871df1715394f92a633b438c5b258de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e41f688ed4d4d768246208cd34e2a49" is_pre_election: true
I20260806 22:42:36.741652 22280 raft_consensus.cc:2454] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7871df1715394f92a633b438c5b258de in term 0.
I20260806 22:42:36.741665 22497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf425c270c1243ffa52d1e319013a224" candidate_uuid: "7871df1715394f92a633b438c5b258de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" is_pre_election: true
I20260806 22:42:36.741809 22497 raft_consensus.cc:2454] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7871df1715394f92a633b438c5b258de in term 0.
I20260806 22:42:36.741883 21960 leader_election.cc:302] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e41f688ed4d4d768246208cd34e2a49, 7871df1715394f92a633b438c5b258de; no voters:
I20260806 22:42:36.742010 22614 raft_consensus.cc:2781] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:36.742087 22614 raft_consensus.cc:482] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:36.742118 22614 raft_consensus.cc:3037] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:36.742815 22614 raft_consensus.cc:504] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.743000 22614 leader_election.cc:288] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 1 election: Requested vote from peers 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643), 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205)
I20260806 22:42:36.743130 22497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf425c270c1243ffa52d1e319013a224" candidate_uuid: "7871df1715394f92a633b438c5b258de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
I20260806 22:42:36.743148 22280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf425c270c1243ffa52d1e319013a224" candidate_uuid: "7871df1715394f92a633b438c5b258de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e41f688ed4d4d768246208cd34e2a49"
I20260806 22:42:36.743219 22497 raft_consensus.cc:3037] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:36.743218 22280 raft_consensus.cc:3037] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:36.744140 22280 raft_consensus.cc:2454] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7871df1715394f92a633b438c5b258de in term 1.
I20260806 22:42:36.744140 22497 raft_consensus.cc:2454] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7871df1715394f92a633b438c5b258de in term 1.
I20260806 22:42:36.744318 21954 leader_election.cc:302] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e86bf4957fb45abb1ffe19e460f23e4, 7871df1715394f92a633b438c5b258de; no voters:
I20260806 22:42:36.744410 22614 raft_consensus.cc:2781] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:36.744567 22614 raft_consensus.cc:686] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 LEADER]: Becoming Leader. State: Replica: 7871df1715394f92a633b438c5b258de, State: Running, Role: LEADER
I20260806 22:42:36.744685 22614 consensus_queue.cc:237] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:36.745458 21808 catalog_manager.cc:5495] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7871df1715394f92a633b438c5b258de (127.0.186.193). New cstate: current_term: 1 leader_uuid: "7871df1715394f92a633b438c5b258de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } health_report { overall_health: UNKNOWN } } }
W20260806 22:42:36.756059 22576 tablet.cc:2369] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:36.766669 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.764498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.0.186.193:36915)
0806 22:42:36.764556 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:36.764558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:36.764566 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:36.764610 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:36.764611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:36.764627 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:36.764689 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:36.764785 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.765272 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.766081 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:36.766261 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:36.766305 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:36.766410 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:36.766510 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:36.766568 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:36.766597 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:42:36.766615 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:42:36.768411 22497 raft_consensus.cc:1261] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Refusing update from remote peer 7871df1715394f92a633b438c5b258de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:36.768415 22280 raft_consensus.cc:1261] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Refusing update from remote peer 7871df1715394f92a633b438c5b258de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:36.768751 22623 consensus_queue.cc:1035] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:36.768875 22614 consensus_queue.cc:1035] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:36.771891 22629 mvcc.cc:205] Tried to move back new op lower bound from 7315686018120916992 to 7315686018026819584. Current Snapshot: MvccSnapshot[applied={T|T < 7315686018120916992}]
I20260806 22:42:36.772607 22630 mvcc.cc:205] Tried to move back new op lower bound from 7315686018120916992 to 7315686018026819584. Current Snapshot: MvccSnapshot[applied={T|T < 7315686018120916992 or (T in {7315686018120916992})}]
I20260806 22:42:36.773394 21809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:36.773505 21809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:36.774230 21809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Sending DeleteTablet for 3 replicas of tablet bf425c270c1243ffa52d1e319013a224
I20260806 22:42:36.774547 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21914
I20260806 22:42:36.774540 22471 tablet_service.cc:1514] Processing DeleteTablet for tablet bf425c270c1243ffa52d1e319013a224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:36 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:48708
I20260806 22:42:36.777097 22247 tablet_service.cc:1514] Processing DeleteTablet for tablet bf425c270c1243ffa52d1e319013a224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:36 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:33900
I20260806 22:42:36.777321 22643 tablet_replica.cc:330] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: stopping tablet replica
I20260806 22:42:36.777401 22643 raft_consensus.cc:2229] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:36.777472 22643 raft_consensus.cc:2258] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:36.780480 22642 tablet_replica.cc:330] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: stopping tablet replica
I20260806 22:42:36.780592 22642 raft_consensus.cc:2229] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:36.780607 22643 ts_tablet_manager.cc:1905] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:36.780685 22642 raft_consensus.cc:2258] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:36.782415 22643 ts_tablet_manager.cc:1918] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:36.782487 22643 log.cc:1196] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/bf425c270c1243ffa52d1e319013a224
I20260806 22:42:36.782698 22643 ts_tablet_manager.cc:1939] T bf425c270c1243ffa52d1e319013a224 P 2e41f688ed4d4d768246208cd34e2a49: Deleting consensus metadata
I20260806 22:42:36.783102 21788 catalog_manager.cc:4841] TS 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): tablet bf425c270c1243ffa52d1e319013a224 (table test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]) successfully deleted
I20260806 22:42:36.784503 22642 ts_tablet_manager.cc:1905] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:36.785796 22642 ts_tablet_manager.cc:1918] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:36.785866 22642 log.cc:1196] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/bf425c270c1243ffa52d1e319013a224
I20260806 22:42:36.786064 22642 ts_tablet_manager.cc:1939] T bf425c270c1243ffa52d1e319013a224 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting consensus metadata
I20260806 22:42:36.786458 21784 catalog_manager.cc:4841] TS 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): tablet bf425c270c1243ffa52d1e319013a224 (table test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]) successfully deleted
W20260806 22:42:36.789048 21788 connection.cc:535] client connection to 127.0.186.193:36915 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:42:36.789160 21788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:42:36.789275 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 22140
I20260806 22:42:36.805016 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:36.804845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:36915 (local address 127.0.0.1:48624)
0806 22:42:36.804908 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:36.804971 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:42:36.805172 21788 catalog_manager.cc:4584] TS 7871df1715394f92a633b438c5b258de (127.0.186.193:36915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf425c270c1243ffa52d1e319013a224: Network error: Client connection negotiation failed: client connection to 127.0.186.193:36915: connect: Connection refused (error 111)
I20260806 22:42:36.808838 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 22359
I20260806 22:42:36.816450 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 21757
I20260806 22:42:36.825346 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--webserver_interface=127.0.186.254
--webserver_port=34255
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:36.905045 22646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:36.905243 22646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:36.905273 22646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:36.906405 22646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:36.906443 22646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:36.906462 22646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:36.906477 22646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:36.906493 22646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:36.907960 22646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=34255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:36.908201 22646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:36.908396 22646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:36.917249 22656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:36.917249 22655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.917323 22646 server_base.cc:1034] running on GCE node
W20260806 22:42:36.917238 22658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:36.917752 22646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:36.918044 22646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:36.919183 22646 hybrid_clock.cc:648] HybridClock initialized: now 1786056156919166 us; error 31 us; skew 500 ppm
Aug 06 22:42:36 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:36.921563 22646 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:36.922019 22646 webserver.cc:460] Webserver started at http://127.0.186.254:34255/ using document root <none> and password file <none>
I20260806 22:42:36.922235 22646 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:36.922289 22646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:36.923617 22646 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:36.924209 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:36.924356 22646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:36.924443 22646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "025c514b9ce84c92854eb4c2a4a30ea8"
format_stamp: "Formatted at 2026-08-06 22:42:35 on dist-test-slave-gjrz"
server_key: "594623052d20fc2a88bbee53cf4d4084"
server_key_iv: "d9c478e6a6d2c7c535d29eb986d7a3e0"
server_key_version: "encryptionkey@0"
I20260806 22:42:36.924825 22646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:36.935175 22646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:36.937175 22646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:36.937306 22646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:36.941169 22646 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:38545
I20260806 22:42:36.941210 22757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:38545 every 8 connection(s)
I20260806 22:42:36.941565 22646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:36.943044 22758 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:36.943754 22758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap starting.
I20260806 22:42:36.950521 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 22646
I20260806 22:42:36.950779 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:36915
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=38627
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:36.952441 22758 log.cc:824] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:36.954523 22758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:36.954854 22758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap complete.
I20260806 22:42:36.958099 22758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.958417 22758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Initialized, Role: FOLLOWER
I20260806 22:42:36.958591 22758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.958673 22758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:36.958725 22758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:36.958776 22758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:42:36.960130 22758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.960244 22758 leader_election.cc:302] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 025c514b9ce84c92854eb4c2a4a30ea8; no voters:
I20260806 22:42:36.960412 22758 leader_election.cc:288] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:42:36.960458 22764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:42:36.960616 22764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 LEADER]: Becoming Leader. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Running, Role: LEADER
I20260806 22:42:36.960672 22758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:36.960783 22764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:36.961089 22764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025c514b9ce84c92854eb4c2a4a30ea8. Latest consensus state: current_term: 2 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:36.961161 22764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:36.961302 22766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:36.961381 22766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:36.961649 22772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:36.962510 22772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:36.962680 22772 catalog_manager.cc:1259] Loaded cluster ID: 69e999ced576448992e259226d8b2133
I20260806 22:42:36.962827 22772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:36.963140 22772 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:42:36.967859 22772 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:42:36.970265 22772 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:36.970484 22772 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Loaded TSK: 0
I20260806 22:42:36.970706 22772 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:42:37.034276 22762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:37.034482 22762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:37.034505 22762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:37.035670 22762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:37.035710 22762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:37.035737 22762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:37.037199 22762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:36915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=38627
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:37.037425 22762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:37.037639 22762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:37.043524 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.043488 22809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.043694 22762 server_base.cc:1034] running on GCE node
W20260806 22:42:37.043502 22806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.043979 22762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:37.044205 22762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:37.045344 22762 hybrid_clock.cc:648] HybridClock initialized: now 1786056157045333 us; error 31 us; skew 500 ppm
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:37.047880 22762 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:37.048313 22762 webserver.cc:460] Webserver started at http://127.0.186.193:38627/ using document root <none> and password file <none>
I20260806 22:42:37.048519 22762 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:37.048590 22762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:37.049865 22762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:37.050490 22818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.050669 22762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.050782 22762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "7871df1715394f92a633b438c5b258de"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "7ea057829a4813544463131ec6b3816b"
server_key_iv: "a5b0523f7dd26ae64972f4d7233fe60f"
server_key_version: "encryptionkey@0"
I20260806 22:42:37.051240 22762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:37.065675 22762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:37.068019 22762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:37.068182 22762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:37.068444 22762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:37.068898 22829 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:42:37.070381 22762 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:42:37.070446 22762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:42:37.070480 22762 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:42:37.071393 22762 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:42:37.071440 22762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:37.071494 22829 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Bootstrap starting.
I20260806 22:42:37.078560 22762 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:36915
I20260806 22:42:37.078718 22987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:36915 every 8 connection(s)
I20260806 22:42:37.079047 22762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:37.081477 22829 log.cc:824] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:37.083738 22829 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:37.083982 22829 tablet_bootstrap.cc:492] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Bootstrap complete.
I20260806 22:42:37.084777 22829 ts_tablet_manager.cc:1397] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.010s
I20260806 22:42:37.085733 22829 raft_consensus.cc:348] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:37.085954 22829 raft_consensus.cc:729] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7871df1715394f92a633b438c5b258de, State: Initialized, Role: FOLLOWER
I20260806 22:42:37.086117 22829 consensus_queue.cc:260] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } } peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } }
I20260806 22:42:37.086359 22829 ts_tablet_manager.cc:1428] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 22:42:37.087037 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 22762
I20260806 22:42:37.087320 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:34643
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=39777
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:37.091840 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.079910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:42609)
0806 22:42:37.080114 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.080169 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.081112 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.081595 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.081600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.081962 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:37.082201 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.082208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.083878 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.083879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.085083 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.085086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.085957 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.085958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.086120 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.086685 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:37.086696 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:37.088450 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:37.090242 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.090249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.090251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.090473 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.090714 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.090715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.090715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.090826 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.091181 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:37.091188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:37.091364 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.091456 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.091646 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260806 22:42:37.091840 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.080265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:42609 (local address 127.0.186.254:38545)
0806 22:42:37.080463 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.080472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.080535 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:42:37.081345 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.081352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.081406 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.081527 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:37.082322 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.083747 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.085185 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.085766 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.085838 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.088569 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:37.088574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:37.088574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:37.088588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:37.090161 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.090549 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.090553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.090553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.090653 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.090883 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.090884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.090884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.091116 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:37.091188 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.091519 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.091646 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260806 22:42:37.092489 22988 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:37.092614 22988 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:37.092834 22988 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:37.093573 22695 ts_manager.cc:194] Registered new tserver with Master: 7871df1715394f92a633b438c5b258de (127.0.186.193:36915)
I20260806 22:42:37.094503 22695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:42609
I20260806 22:42:37.095541 22988 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
I20260806 22:42:37.103592 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.094206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:36915 (local address 127.0.0.1:48640)
0806 22:42:37.099380 (+ 5174us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.099421 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.099537 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.099768 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.099771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.099773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.099857 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.099859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.101122 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.101123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.102349 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.102353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.103344 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.103345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.103416 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.103428 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.103486 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.103529 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5138,"thread_start_us":92,"threads_started":1}
I20260806 22:42:37.103629 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.094266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.0.186.193:36915)
0806 22:42:37.094795 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.094799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.099495 (+ 4696us) server_negotiation.cc:407] Connection header received
0806 22:42:37.099649 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.099653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.099671 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.099761 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.099921 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.101035 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.102482 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.103225 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.103339 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.103403 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.103512 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.103552 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":80,"threads_started":1}
I20260806 22:42:37.104012 22893 tablet_service.cc:1514] Processing DeleteTablet for tablet bf425c270c1243ffa52d1e319013a224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:36 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:48640
I20260806 22:42:37.104528 23003 tablet_replica.cc:330] stopping tablet replica
I20260806 22:42:37.104645 23003 raft_consensus.cc:2229] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:37.104715 23003 raft_consensus.cc:2258] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:37.105051 23003 ts_tablet_manager.cc:1905] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:37.107384 23003 ts_tablet_manager.cc:1918] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:37.107465 23003 log.cc:1196] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/bf425c270c1243ffa52d1e319013a224
I20260806 22:42:37.107692 23003 ts_tablet_manager.cc:1939] T bf425c270c1243ffa52d1e319013a224 P 7871df1715394f92a633b438c5b258de: Deleting consensus metadata
I20260806 22:42:37.108228 22673 catalog_manager.cc:4841] TS 7871df1715394f92a633b438c5b258de (127.0.186.193:36915): tablet bf425c270c1243ffa52d1e319013a224 (table test-table [id=12e4f0e358f44663b56e1ea7b12b8c73]) successfully deleted
W20260806 22:42:37.173780 22999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:37.173976 22999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:37.174001 22999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:37.175113 22999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:37.175153 22999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:37.175181 22999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:37.176630 22999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:34643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=39777
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:37.176838 22999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:37.177045 22999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:37.181708 23010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.181841 22999 server_base.cc:1034] running on GCE node
W20260806 22:42:37.181740 23013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.181773 23011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.182145 22999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:37.182375 22999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:37.183509 22999 hybrid_clock.cc:648] HybridClock initialized: now 1786056157183494 us; error 25 us; skew 500 ppm
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:37.185917 22999 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:37.186301 22999 webserver.cc:460] Webserver started at http://127.0.186.194:39777/ using document root <none> and password file <none>
I20260806 22:42:37.186511 22999 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:37.186574 22999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:37.187947 22999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:37.188519 23025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.188673 22999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:37.188753 22999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "2e41f688ed4d4d768246208cd34e2a49"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5489c894468f538c1426443e7750c631"
server_key_iv: "5e607f0a314a5da3ad62e4ad0bbf5ea2"
server_key_version: "encryptionkey@0"
I20260806 22:42:37.189069 22999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:37.200873 22999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:37.202513 22999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:37.202714 22999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:37.202919 22999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:37.203192 22999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:37.203224 22999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.203250 22999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:37.203312 22999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.208616 22999 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34643
I20260806 22:42:37.208678 23197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34643 every 8 connection(s)
I20260806 22:42:37.208954 22999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:37.214499 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 22999
I20260806 22:42:37.214752 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:35205
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=45673
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:37.217258 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.209601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:46073 (local address 127.0.186.254:38545)
0806 22:42:37.209645 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.209647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.209828 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:42:37.210281 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.210282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.210295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.210334 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:37.211184 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.211738 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.212637 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.212826 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.212868 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.214792 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:37.214793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:37.214794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:37.214801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:37.216046 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.216396 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.216398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.216398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.216454 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.216667 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.216668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.216668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.216774 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:37.216829 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.217078 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.217170 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:37.217391 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.209576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:46073)
0806 22:42:37.209721 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.209756 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.210192 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.210387 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.210390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.210934 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:37.211113 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.211117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.211803 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.211804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.212574 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.212577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.212877 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.212877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.212976 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.213402 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:37.213409 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:37.214705 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:37.216123 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.216130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.216132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.216330 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.216508 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.216509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.216509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.216612 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.216831 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:37.216837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:37.216964 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.217049 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.217197 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 22:42:37.217857 23198 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:37.217947 23198 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:37.218112 23198 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:37.218487 22695 ts_manager.cc:194] Registered new tserver with Master: 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
I20260806 22:42:37.218845 22695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:46073
W20260806 22:42:37.296252 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:37.296442 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:37.296463 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:37.297588 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:37.297631 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:37.297658 23201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:37.299064 23201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:35205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=45673
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:37.299285 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:37.299491 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:37.304270 23213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.304347 23201 server_base.cc:1034] running on GCE node
W20260806 22:42:37.304270 23219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.304312 23214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.304670 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:37.304893 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:37.306049 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786056157306032 us; error 32 us; skew 500 ppm
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:37.308594 23201 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:37.309023 23201 webserver.cc:460] Webserver started at http://127.0.186.195:45673/ using document root <none> and password file <none>
I20260806 22:42:37.309218 23201 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:37.309280 23201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:37.310528 23201 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:37.311045 23228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.311220 23201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:37.311324 23201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5e2b890a3bd9e257539c951e241c7cfa"
server_key_iv: "96e1cc33232acffe68bc0957eb52a3e8"
server_key_version: "encryptionkey@0"
I20260806 22:42:37.311638 23201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:37.318054 23201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:37.319700 23201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:37.319824 23201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:37.320019 23201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:37.320320 23201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:37.320356 23201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.320382 23201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:37.320406 23201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.325922 23201 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35205
I20260806 22:42:37.325986 23396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35205 every 8 connection(s)
I20260806 22:42:37.326292 23201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:37.330428 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 23201
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:37.335668 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.326959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:53555 (local address 127.0.186.254:38545)
0806 22:42:37.327001 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.327003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.327255 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:42:37.327760 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.327761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.327774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.327826 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:37.328781 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.329270 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.330212 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.330410 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.330465 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.332734 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:37.332736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:37.332736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:37.332744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:37.334325 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.334699 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.334700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.334701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.334758 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.334966 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.334967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.334967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.335091 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:37.335150 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.335442 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.335563 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:37.335770 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.326967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:53555)
0806 22:42:37.327142 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.327176 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.327666 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.327891 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.327895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.328489 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:37.328684 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.328688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.329352 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.329353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.330132 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.330134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.330478 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.330478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.330631 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.331183 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:37.331193 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:37.332646 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:37.334406 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.334412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.334414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.334624 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.334812 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:37.334813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:37.334813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:37.334909 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:37.335148 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:37.335154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:37.335302 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.335399 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.335557 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260806 22:42:37.336172 23397 heartbeater.cc:347] Connected to a master server at 127.0.186.254:38545
I20260806 22:42:37.336294 23397 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:37.336483 23397 heartbeater.cc:511] Master 127.0.186.254:38545 requested a full tablet report, sending...
I20260806 22:42:37.336885 22695 ts_manager.cc:194] Registered new tserver with Master: 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205)
I20260806 22:42:37.337132 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.330784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.0.186.254:38545)
0806 22:42:37.330889 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.330891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.331022 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:42:37.331087 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.331088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.331099 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.331159 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:37.331722 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.332239 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.333055 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.333335 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.333385 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.334596 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:37.334597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:37.334598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:37.334604 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:37.336059 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.336410 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.336411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.336411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.336463 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:37.336644 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:37.336645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:37.336645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:37.336767 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:37.336927 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.336965 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.337044 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
I20260806 22:42:37.337393 22695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:53555
I20260806 22:42:37.337823 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:37.340301 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.338134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.0.186.254:38545)
0806 22:42:37.338176 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.338177 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.338241 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:42:37.338305 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.338306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.338332 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.338381 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:37.338460 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.338893 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.339688 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.339867 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.339914 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.340067 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:37.340153 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:37.340207 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.340239 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.340256 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:37.340543 22695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:37.342079 22695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:37.344084 22893 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 (DEFAULT_TABLE table=test-table [id=27c1b3e6670f46a596d9046f3549d827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:37.344259 22893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1ec9e57b664ef6a23a1a0979715423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:37.345611 22829 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Bootstrap starting.
I20260806 22:42:37.345950 22829 tablet_bootstrap.cc:654] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:37.346531 22829 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: No bootstrap required, opened a new log
I20260806 22:42:37.346580 22829 ts_tablet_manager.cc:1397] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:42:37.346679 22829 raft_consensus.cc:348] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.346740 22829 raft_consensus.cc:374] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:37.346779 22829 raft_consensus.cc:729] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7871df1715394f92a633b438c5b258de, State: Initialized, Role: FOLLOWER
I20260806 22:42:37.346819 22829 consensus_queue.cc:260] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.346890 22829 ts_tablet_manager.cc:1428] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.347059 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.343743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34643 (local address 127.0.0.1:33902)
0806 22:42:37.343831 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.343842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.343886 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.344295 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.344296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.344298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.344334 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.344336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.345339 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.345340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.346077 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.346089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.346844 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.346845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.346906 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.346910 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.346954 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.346997 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 22:42:37.347314 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.343975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.0.186.194:34643)
0806 22:42:37.344142 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.344146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.344157 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:37.344196 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.344198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.344213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.344289 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.344386 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.345255 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.346165 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.346774 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.346835 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.346893 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.347170 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.347211 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260806 22:42:37.348225 23100 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 (DEFAULT_TABLE table=test-table [id=27c1b3e6670f46a596d9046f3549d827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:37.348532 23100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1ec9e57b664ef6a23a1a0979715423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:37.349102 23407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.343745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35205 (local address 127.0.0.1:48718)
0806 22:42:37.343911 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.343921 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.343967 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.345941 (+ 1974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.345942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.345944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.345985 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.345987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.347147 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.347148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.348146 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.348149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.348865 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.348866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.348929 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.348932 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.348984 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.349030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":18,"thread_start_us":89,"threads_started":1}
I20260806 22:42:37.349110 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.0.186.195:35205)
0806 22:42:37.345730 (+ 1894us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.345735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.345746 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:37.345786 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.345788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.345802 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.345874 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.346084 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.347041 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.348215 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.348791 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.348853 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.348908 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.348998 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.349036 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1858,"thread_start_us":82,"threads_started":1}
I20260806 22:42:37.349828 23299 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 (DEFAULT_TABLE table=test-table [id=27c1b3e6670f46a596d9046f3549d827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:37.350162 23299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1ec9e57b664ef6a23a1a0979715423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:37.351785 23412 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Bootstrap starting.
I20260806 22:42:37.352392 23412 tablet_bootstrap.cc:654] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:37.352864 23412 log.cc:824] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:37.353112 23413 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Bootstrap starting.
I20260806 22:42:37.353554 23412 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: No bootstrap required, opened a new log
I20260806 22:42:37.353610 23412 ts_tablet_manager.cc:1397] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:37.353713 23413 tablet_bootstrap.cc:654] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:37.354152 23413 log.cc:824] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:37.354851 23413 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: No bootstrap required, opened a new log
I20260806 22:42:37.354918 23413 ts_tablet_manager.cc:1397] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:37.355317 23412 raft_consensus.cc:348] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.355459 23412 raft_consensus.cc:374] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:37.355492 23412 raft_consensus.cc:729] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e41f688ed4d4d768246208cd34e2a49, State: Initialized, Role: FOLLOWER
I20260806 22:42:37.355615 23412 consensus_queue.cc:260] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.355902 23412 ts_tablet_manager.cc:1428] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:37.355906 23198 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
I20260806 22:42:37.356490 23413 raft_consensus.cc:348] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.356652 23413 raft_consensus.cc:374] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:37.356700 23413 raft_consensus.cc:729] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e86bf4957fb45abb1ffe19e460f23e4, State: Initialized, Role: FOLLOWER
I20260806 22:42:37.356814 23413 consensus_queue.cc:260] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.357090 23413 ts_tablet_manager.cc:1428] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:37.357112 23397 heartbeater.cc:503] Master 127.0.186.254:38545 was elected leader, sending a full tablet report...
W20260806 22:42:37.459473 23199 tablet.cc:2369] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:37.576838 23399 tablet.cc:2369] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:37.582198 22990 tablet.cc:2369] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:37.619801 23417 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:37.619930 23417 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.620261 23417 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 7871df1715394f92a633b438c5b258de (127.0.186.193:36915)
I20260806 22:42:37.622987 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.620535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45879 (local address 127.0.186.195:35205)
0806 22:42:37.620604 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.620606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.620743 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:42:37.620796 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.620797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.620806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.620854 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.620987 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.621443 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622327 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.622692 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622751 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.622761 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.622883 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.622919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:42:37.623004 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.620467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35205 (local address 127.0.186.194:45879)
0806 22:42:37.620671 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.620679 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.620738 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.620860 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.620861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.620862 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.620932 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.620934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.621507 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.621508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.622246 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.622248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.622757 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.622822 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.622825 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.622870 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.622911 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":129,"threads_started":1}
I20260806 22:42:37.623014 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.620552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50279 (local address 127.0.186.193:36915)
0806 22:42:37.620679 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.620681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.620688 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:37.620713 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.620714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.620723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.620776 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:37.620919 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.621408 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622318 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.622694 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622754 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.622763 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.622897 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.622946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260806 22:42:37.623013 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.620454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:36915 (local address 127.0.186.194:50279)
0806 22:42:37.620529 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.620542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:37.620596 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:37.620788 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:37.620789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:37.620791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:37.620857 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.620858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.621482 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.621483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.622228 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:37.622230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.622761 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.622762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:37.622833 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.622836 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:37.622881 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:42:37.622924 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":12}
I20260806 22:42:37.623391 23327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1ec9e57b664ef6a23a1a0979715423" candidate_uuid: "2e41f688ed4d4d768246208cd34e2a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" is_pre_election: true
I20260806 22:42:37.623383 22927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1ec9e57b664ef6a23a1a0979715423" candidate_uuid: "2e41f688ed4d4d768246208cd34e2a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7871df1715394f92a633b438c5b258de" is_pre_election: true
I20260806 22:42:37.623553 22927 raft_consensus.cc:2454] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e41f688ed4d4d768246208cd34e2a49 in term 0.
I20260806 22:42:37.623553 23327 raft_consensus.cc:2454] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e41f688ed4d4d768246208cd34e2a49 in term 0.
I20260806 22:42:37.623773 23026 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4; no voters:
I20260806 22:42:37.623944 23417 raft_consensus.cc:2781] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:37.624006 23417 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:37.624022 23417 raft_consensus.cc:3037] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:37.624706 23417 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.624827 23417 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [CANDIDATE]: Term 1 election: Requested vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 7871df1715394f92a633b438c5b258de (127.0.186.193:36915)
I20260806 22:42:37.624966 23327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1ec9e57b664ef6a23a1a0979715423" candidate_uuid: "2e41f688ed4d4d768246208cd34e2a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
I20260806 22:42:37.624989 22927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1ec9e57b664ef6a23a1a0979715423" candidate_uuid: "2e41f688ed4d4d768246208cd34e2a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7871df1715394f92a633b438c5b258de"
I20260806 22:42:37.625049 23327 raft_consensus.cc:3037] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:37.625066 22927 raft_consensus.cc:3037] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:37.625757 23327 raft_consensus.cc:2454] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e41f688ed4d4d768246208cd34e2a49 in term 1.
I20260806 22:42:37.625801 22927 raft_consensus.cc:2454] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e41f688ed4d4d768246208cd34e2a49 in term 1.
I20260806 22:42:37.625922 23026 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4; no voters:
I20260806 22:42:37.626015 23417 raft_consensus.cc:2781] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:37.626166 23417 raft_consensus.cc:686] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 LEADER]: Becoming Leader. State: Replica: 2e41f688ed4d4d768246208cd34e2a49, State: Running, Role: LEADER
I20260806 22:42:37.626259 23417 consensus_queue.cc:237] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:37.627102 22695 catalog_manager.cc:5495] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "2e41f688ed4d4d768246208cd34e2a49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } health_report { overall_health: UNKNOWN } } }
I20260806 22:42:37.700225 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.698101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.0.186.194:34643)
0806 22:42:37.698147 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:37.698149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:37.698160 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:42:37.698203 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:37.698204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:37.698222 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:37.698299 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:37.698390 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.698850 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.699637 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:37.699824 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:37.699874 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:37.699975 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:37.700075 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:37.700136 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:37.700148 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 22:42:37.700165 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:37.701921 23327 raft_consensus.cc:1261] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Refusing update from remote peer 2e41f688ed4d4d768246208cd34e2a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:37.701931 22927 raft_consensus.cc:1261] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Refusing update from remote peer 2e41f688ed4d4d768246208cd34e2a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:37.702251 23417 consensus_queue.cc:1035] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:37.702392 23427 consensus_queue.cc:1035] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:37.705011 23434 mvcc.cc:205] Tried to move back new op lower bound from 7315686021944389632 to 7315686021637623808. Current Snapshot: MvccSnapshot[applied={T|T < 7315686021944389632}]
I20260806 22:42:37.706763 22695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:37.706864 22695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:37.707576 22695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Sending DeleteTablet for 3 replicas of tablet 6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:37.707882 23299 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:37 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:48718
I20260806 22:42:37.707976 23100 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:37 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:33902
I20260806 22:42:37.708040 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 22762
I20260806 22:42:37.708072 22893 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1ec9e57b664ef6a23a1a0979715423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:37 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:48640
I20260806 22:42:37.712018 23442 tablet_replica.cc:330] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: stopping tablet replica
I20260806 22:42:37.712203 23442 raft_consensus.cc:2229] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:37.712391 23442 raft_consensus.cc:2258] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:37.714135 23442 ts_tablet_manager.cc:1905] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:37.716483 23442 ts_tablet_manager.cc:1918] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:37.716588 23442 log.cc:1196] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:37.717015 23442 ts_tablet_manager.cc:1939] T 6c1ec9e57b664ef6a23a1a0979715423 P 5e86bf4957fb45abb1ffe19e460f23e4: Deleting consensus metadata
I20260806 22:42:37.717555 22672 catalog_manager.cc:4841] TS 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): tablet 6c1ec9e57b664ef6a23a1a0979715423 (table test-table [id=27c1b3e6670f46a596d9046f3549d827]) successfully deleted
I20260806 22:42:37.721095 23443 tablet_replica.cc:330] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: stopping tablet replica
I20260806 22:42:37.721233 23443 raft_consensus.cc:2229] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:37.721351 23443 raft_consensus.cc:2258] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:37.721825 23443 ts_tablet_manager.cc:1905] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:37.723423 23443 ts_tablet_manager.cc:1918] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:37.723500 23443 log.cc:1196] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:37.723812 23443 ts_tablet_manager.cc:1939] T 6c1ec9e57b664ef6a23a1a0979715423 P 2e41f688ed4d4d768246208cd34e2a49: Deleting consensus metadata
I20260806 22:42:37.724259 22673 catalog_manager.cc:4841] TS 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): tablet 6c1ec9e57b664ef6a23a1a0979715423 (table test-table [id=27c1b3e6670f46a596d9046f3549d827]) successfully deleted
W20260806 22:42:37.725754 22673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:36915 (error 108)
I20260806 22:42:37.726064 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 22999
I20260806 22:42:37.727741 23407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:37.726204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:36915 (local address 127.0.0.1:48650)
0806 22:42:37.727662 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:37.727704 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1437}
W20260806 22:42:37.727860 22673 catalog_manager.cc:4584] TS 7871df1715394f92a633b438c5b258de (127.0.186.193:36915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c1ec9e57b664ef6a23a1a0979715423: Network error: Client connection negotiation failed: client connection to 127.0.186.193:36915: connect: Connection refused (error 111)
I20260806 22:42:37.733764 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 23201
I20260806 22:42:37.741909 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 22646
I20260806 22:42:37.751027 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--webserver_interface=127.0.186.254
--webserver_port=34255
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:37.835157 23448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:37.835397 23448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:37.835433 23448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:37.836550 23448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:37.836589 23448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:37.836609 23448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:37.836624 23448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:37.836640 23448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:37.838239 23448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=34255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:37.838490 23448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:37.838689 23448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:37.843364 23461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.843369 23462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.843391 23464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.843415 23448 server_base.cc:1034] running on GCE node
I20260806 22:42:37.843787 23448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:37.844022 23448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:37.845162 23448 hybrid_clock.cc:648] HybridClock initialized: now 1786056157845150 us; error 29 us; skew 500 ppm
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:37.847545 23448 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:37.847936 23448 webserver.cc:460] Webserver started at http://127.0.186.254:34255/ using document root <none> and password file <none>
I20260806 22:42:37.848155 23448 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:37.848201 23448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:37.849422 23448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:37.850060 23471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.850206 23448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:37.850289 23448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "025c514b9ce84c92854eb4c2a4a30ea8"
format_stamp: "Formatted at 2026-08-06 22:42:35 on dist-test-slave-gjrz"
server_key: "594623052d20fc2a88bbee53cf4d4084"
server_key_iv: "d9c478e6a6d2c7c535d29eb986d7a3e0"
server_key_version: "encryptionkey@0"
I20260806 22:42:37.850602 23448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:37.870607 23448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:37.872351 23448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:37.872484 23448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:37.876296 23448 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:38545
I20260806 22:42:37.876331 23547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:38545 every 8 connection(s)
I20260806 22:42:37.876674 23448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:37.877585 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 23448
I20260806 22:42:37.877856 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:36915
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=38627
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:37.878319 23548 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:42:37.878950 23548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap starting.
I20260806 22:42:37.888314 23548 log.cc:824] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:37.890453 23548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:37.890714 23548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8: Bootstrap complete.
I20260806 22:42:37.892804 23548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:37.893019 23548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Initialized, Role: FOLLOWER
I20260806 22:42:37.893167 23548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:37.893231 23548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:37.893272 23548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:37.893316 23548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:42:37.894250 23548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:37.894338 23548 leader_election.cc:302] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 025c514b9ce84c92854eb4c2a4a30ea8; no voters:
I20260806 22:42:37.894500 23548 leader_election.cc:288] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:42:37.894562 23552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:42:37.894687 23552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [term 3 LEADER]: Becoming Leader. State: Replica: 025c514b9ce84c92854eb4c2a4a30ea8, State: Running, Role: LEADER
I20260806 22:42:37.894760 23548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:37.894771 23552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } }
I20260806 22:42:37.895140 23554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:37.895151 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025c514b9ce84c92854eb4c2a4a30ea8. Latest consensus state: current_term: 3 leader_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025c514b9ce84c92854eb4c2a4a30ea8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38545 } } }
I20260806 22:42:37.895299 23554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:37.895388 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025c514b9ce84c92854eb4c2a4a30ea8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:37.895920 23564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:37.896716 23564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:37.896869 23564 catalog_manager.cc:1259] Loaded cluster ID: 69e999ced576448992e259226d8b2133
I20260806 22:42:37.897092 23564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:42:37.897435 23564 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f02ee6db9ad google::LogMessage::Fail() at ??:0
@ 0x7f02ee6de88a google::LogMessage::SendToLog() at ??:0
@ 0x7f02ee6db4de google::LogMessage::Flush() at ??:0
@ 0x7f02ee6dca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f02f65fc0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f02f66058b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f02f05a90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f02f059e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f02f14026db start_thread at ??:0
@ 0x7f02ed1d961f clone at ??:0
W20260806 22:42:37.988338 23550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:37.988617 23550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:37.988659 23550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:37.990334 23550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:37.990393 23550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:37.990443 23550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:37.992628 23550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:36915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=38627
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:37.992887 23550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:37.993149 23550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:37.998275 23600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.998303 23598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:37.998381 23602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:37.998595 23550 server_base.cc:1034] running on GCE node
I20260806 22:42:37.998909 23550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:37.999128 23550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:38.000268 23550 hybrid_clock.cc:648] HybridClock initialized: now 1786056158000254 us; error 35 us; skew 500 ppm
Aug 06 22:42:37 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:38.002836 23550 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:38.003361 23550 webserver.cc:460] Webserver started at http://127.0.186.193:38627/ using document root <none> and password file <none>
I20260806 22:42:38.003630 23550 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:38.003688 23550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:38.005405 23550 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:38.006240 23611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.006420 23550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:38.006506 23550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "7871df1715394f92a633b438c5b258de"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "7ea057829a4813544463131ec6b3816b"
server_key_iv: "a5b0523f7dd26ae64972f4d7233fe60f"
server_key_version: "encryptionkey@0"
I20260806 22:42:38.006843 23550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:38.029668 23550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:38.031859 23550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:38.031999 23550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:38.032222 23550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:38.032680 23622 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:42:38.034643 23550 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:42:38.034690 23550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:42:38.034723 23550 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:42:38.035403 23550 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:42:38.035437 23550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:38.035547 23622 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Bootstrap starting.
I20260806 22:42:38.042380 23550 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:36915
I20260806 22:42:38.042498 23798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:36915 every 8 connection(s)
I20260806 22:42:38.042810 23550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:42:38.044072 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 23550
I20260806 22:42:38.044366 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:34643
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=39777
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:38.046973 23622 log.cc:824] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:38.049119 23622 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:42:38.049410 23622 tablet_bootstrap.cc:492] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Bootstrap complete.
I20260806 22:42:38.050218 23622 ts_tablet_manager.cc:1397] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260806 22:42:38.051225 23622 raft_consensus.cc:348] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:38.051505 23622 raft_consensus.cc:729] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7871df1715394f92a633b438c5b258de, State: Initialized, Role: FOLLOWER
I20260806 22:42:38.051676 23622 consensus_queue.cc:260] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:38.051954 23622 ts_tablet_manager.cc:1428] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260806 22:42:38.125715 23806 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:38.043478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:38187)
0806 22:42:38.043653 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.043696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:38.044303 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:38.125505 (+ 81202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
W20260806 22:42:38.125993 23801 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:38545 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:38545: Client connection negotiation failed: client connection to 127.0.186.254:38545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:42:38.126367 23806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.126227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:49205)
0806 22:42:38.126280 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.126333 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:42:38.126776 23806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.126688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:50559)
0806 22:42:38.126731 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.126757 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:42:38.126894 23801 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:42:38.149905 23808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:38.150087 23808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:38.150118 23808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:38.151567 23808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:38.151614 23808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:38.151650 23808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:38.153065 23808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:34643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=39777
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:38.153292 23808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:38.153502 23808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:38.158142 23823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:38.158150 23819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:38.158253 23820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:38.158388 23808 server_base.cc:1034] running on GCE node
I20260806 22:42:38.158680 23808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:38.158902 23808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:38.160086 23808 hybrid_clock.cc:648] HybridClock initialized: now 1786056158160065 us; error 28 us; skew 500 ppm
Aug 06 22:42:38 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:38.162496 23808 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:38.162891 23808 webserver.cc:460] Webserver started at http://127.0.186.194:39777/ using document root <none> and password file <none>
I20260806 22:42:38.163121 23808 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:38.163173 23808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:38.164461 23808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:38.165067 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.165242 23808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:38.165330 23808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "2e41f688ed4d4d768246208cd34e2a49"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5489c894468f538c1426443e7750c631"
server_key_iv: "5e607f0a314a5da3ad62e4ad0bbf5ea2"
server_key_version: "encryptionkey@0"
I20260806 22:42:38.165640 23808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:38.182332 23808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:38.183997 23808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:38.184129 23808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:38.184331 23808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:38.184613 23808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:38.184646 23808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.184674 23808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:38.184691 23808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.190241 23808 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:34643
I20260806 22:42:38.190313 24023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:34643 every 8 connection(s)
I20260806 22:42:38.190613 23808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:42:38.191577 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.191224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:44789)
0806 22:42:38.191378 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.191443 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
W20260806 22:42:38.191942 24024 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:38545 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:38545: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
I20260806 22:42:38.192196 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.192117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:58037)
0806 22:42:38.192154 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.192180 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 22:42:38.192345 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 23808
I20260806 22:42:38.192478 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.192395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:39167)
0806 22:42:38.192435 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.192458 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:42:38.192579 24024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:42:38.192637 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:35205
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:38545
--webserver_port=45673
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:38.273880 24030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:38.274075 24030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:38.274096 24030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:38.275221 24030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:38.275280 24030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:38.275326 24030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:38.276748 24030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:35205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=45673
--tserver_master_addrs=127.0.186.254:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:38.276953 24030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:38.277163 24030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:38.282011 24036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:38.282017 24035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:38.282038 24030 server_base.cc:1034] running on GCE node
W20260806 22:42:38.282011 24038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:38.282500 24030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:38.282732 24030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:38.283871 24030 hybrid_clock.cc:648] HybridClock initialized: now 1786056158283857 us; error 26 us; skew 500 ppm
Aug 06 22:42:38 dist-test-slave-gjrz krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:38.291474 24030 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:38.291909 24030 webserver.cc:460] Webserver started at http://127.0.186.195:45673/ using document root <none> and password file <none>
I20260806 22:42:38.292114 24030 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:38.292161 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:38.293479 24030 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:38.294086 24047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.294250 24030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:38.294335 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "5e86bf4957fb45abb1ffe19e460f23e4"
format_stamp: "Formatted at 2026-08-06 22:42:36 on dist-test-slave-gjrz"
server_key: "5e2b890a3bd9e257539c951e241c7cfa"
server_key_iv: "96e1cc33232acffe68bc0957eb52a3e8"
server_key_version: "encryptionkey@0"
I20260806 22:42:38.294664 24030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:38.338843 24030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:38.341110 24030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:38.341257 24030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:38.341492 24030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:38.341806 24030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:38.341841 24030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.341882 24030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:38.341908 24030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:38.347242 24030 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:35205
I20260806 22:42:38.347337 24230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:35205 every 8 connection(s)
I20260806 22:42:38.347666 24030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:38.348376 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 24030
I20260806 22:42:38.348724 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.348304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:46289)
0806 22:42:38.348475 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.348573 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
W20260806 22:42:38.349124 24231 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:38545 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:38545: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
I20260806 22:42:38.349390 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.349302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:58927)
0806 22:42:38.349344 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.349369 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:42:38.349635 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:38.349562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:38507)
0806 22:42:38.349594 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:38.349616 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 22:42:38.349728 24231 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:42:39.127525 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.127241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:51577)
0806 22:42:39.127392 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:39.127446 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260806 22:42:39.193100 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.192886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:35397)
0806 22:42:39.193009 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:39.193045 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260806 22:42:39.304498 24242 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:39.304637 24242 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:39.304981 24242 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
Aug 06 22:42:39 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.194@KRBTEST.COM
Aug 06 22:42:39 dist-test-slave-gjrz krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.195@KRBTEST.COM
I20260806 22:42:39.314949 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.305202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:36371 (local address 127.0.186.195:35205)
0806 22:42:39.305376 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:39.305389 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:39.305415 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:42:39.305856 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:39.305859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:39.305910 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:39.306029 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:39.306802 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.307815 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.308801 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.309182 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.309267 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:39.311963 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:39.311967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:39.311967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:39.311979 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:39.313457 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313817 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:39.313820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:39.313820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:39.313910 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:39.314143 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:39.314143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:39.314144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:39.314378 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:39.314450 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:39.314729 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:42:39.314851 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260806 22:42:39.314965 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.305309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:46769 (local address 127.0.186.194:34643)
0806 22:42:39.305421 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:39.305430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:39.305459 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:42:39.305884 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:39.305886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:39.305922 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:39.306051 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:39.306816 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.307785 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.308800 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.309182 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.309309 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:39.311618 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:39.311622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:39.311622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:39.311632 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:39.313385 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313821 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:39.313824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:39.313824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:39.313913 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:39.314147 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:39.314148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:39.314149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:39.314378 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:39.314450 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:39.314729 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:42:39.314845 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":51,"threads_started":1}
I20260806 22:42:39.314975 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.305199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:34643 (local address 127.0.186.193:46769)
0806 22:42:39.305302 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:39.305311 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:39.305367 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:39.306101 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:39.306104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:39.306506 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:39.306685 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:39.306692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.307873 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.307874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:39.308717 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:39.308719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.309282 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.309283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:39.309400 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:39.309973 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:39.309978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:39.311497 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:39.313453 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313461 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:39.313463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:39.313748 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:39.313973 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:39.313974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:39.314090 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:39.314444 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:39.314451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:39.314579 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 22:42:39.314667 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:42:39.314854 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":50,"thread_start_us":53,"threads_started":1}
I20260806 22:42:39.315064 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.305108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:35205 (local address 127.0.186.193:36371)
0806 22:42:39.305163 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:39.305175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:39.305234 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:39.306101 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:39.306104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:39.306505 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:39.306685 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:39.306692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.307899 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.307900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:39.308718 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:39.308721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:39.309279 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:39.309280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:39.309402 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:39.309953 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:39.309963 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:39.311871 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:39.313520 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:39.313522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:39.313748 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:39.313967 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:39.313968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:39.313969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:39.314087 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:39.314443 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:39.314450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:39.314579 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:42:39.314667 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:42:39.314927 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":182}
W20260806 22:42:39.315807 23615 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
W20260806 22:42:39.316041 23612 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:39.316093 23612 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7871df1715394f92a633b438c5b258de; no voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4
I20260806 22:42:39.316180 24242 raft_consensus.cc:2726] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:42:39.350231 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:39.350028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:46297)
0806 22:42:39.350148 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:39.350192 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260806 22:42:40.128162 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.127952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:41615)
0806 22:42:40.128076 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.128114 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260806 22:42:40.193755 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.193526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:57263)
0806 22:42:40.193675 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.193713 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260806 22:42:40.350876 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:40.350675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:37481)
0806 22:42:40.350794 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:40.350837 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260806 22:42:41.128829 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.128600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:58625)
0806 22:42:41.128734 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.128774 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260806 22:42:41.194397 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.194193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:48961)
0806 22:42:41.194319 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.194356 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 22:42:41.351544 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:41.351335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:60213)
0806 22:42:41.351463 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:41.351504 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260806 22:42:41.408803 24269 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:41.408898 24269 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:41.409089 24269 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
W20260806 22:42:41.409533 23615 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
W20260806 22:42:41.409627 23612 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:41.409675 23612 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7871df1715394f92a633b438c5b258de; no voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4
I20260806 22:42:41.409747 24269 raft_consensus.cc:2726] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:42:42.129480 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:42.129253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:41009)
0806 22:42:42.129389 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:42.129430 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260806 22:42:42.195010 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:42.194806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:56249)
0806 22:42:42.194935 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:42.194972 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260806 22:42:42.352181 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:42.351997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:43161)
0806 22:42:42.352107 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:42.352144 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260806 22:42:43.130266 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:43.130022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:55113)
0806 22:42:43.130175 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:43.130216 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260806 22:42:43.195658 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:43.195431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:48867)
0806 22:42:43.195579 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:43.195617 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":55,"threads_started":1}
I20260806 22:42:43.352778 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:43.352579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:38939)
0806 22:42:43.352699 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:43.352737 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260806 22:42:44.131012 24281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.130758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:57721)
0806 22:42:44.130917 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.130960 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260806 22:42:44.196246 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.196037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:50367)
0806 22:42:44.196164 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.196203 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260806 22:42:44.237812 24284 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:44.237914 24284 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:44.238093 24284 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
W20260806 22:42:44.238494 23615 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
W20260806 22:42:44.238595 23612 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:44.238660 23612 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7871df1715394f92a633b438c5b258de; no voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4
I20260806 22:42:44.238732 24284 raft_consensus.cc:2726] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:42:44.353425 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:44.353220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:58333)
0806 22:42:44.353344 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:44.353382 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 22:42:45.131713 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.131503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:55135)
0806 22:42:45.131626 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.131664 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260806 22:42:45.196851 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.196658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:60965)
0806 22:42:45.196775 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.196808 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260806 22:42:45.354044 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:45.353837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:57739)
0806 22:42:45.353969 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:45.354005 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260806 22:42:46.132375 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.132147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:45169)
0806 22:42:46.132289 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.132326 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260806 22:42:46.197443 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.197224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:46541)
0806 22:42:46.197363 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.197400 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260806 22:42:46.354655 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:46.354434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:36145)
0806 22:42:46.354575 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:46.354616 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260806 22:42:47.132999 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.132788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:34387)
0806 22:42:47.132909 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.132950 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260806 22:42:47.198076 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.197848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:43889)
0806 22:42:47.197981 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.198027 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260806 22:42:47.355456 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:47.355179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:56749)
0806 22:42:47.355335 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:47.355400 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260806 22:42:48.133759 24296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.133507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:35531)
0806 22:42:48.133663 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.133710 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260806 22:42:48.198664 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.198465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:42729)
0806 22:42:48.198586 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.198624 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260806 22:42:48.356163 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:48.355939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:44947)
0806 22:42:48.356083 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:48.356122 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":62,"threads_started":1}
I20260806 22:42:48.717572 24300 raft_consensus.cc:482] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:48.717674 24300 raft_consensus.cc:504] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e86bf4957fb45abb1ffe19e460f23e4" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 35205 } } peers { permanent_uuid: "2e41f688ed4d4d768246208cd34e2a49" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 34643 } } peers { permanent_uuid: "7871df1715394f92a633b438c5b258de" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 36915 } }
I20260806 22:42:48.717847 24300 leader_election.cc:288] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205), 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643)
W20260806 22:42:48.718207 23615 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e41f688ed4d4d768246208cd34e2a49 (127.0.186.194:34643): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
W20260806 22:42:48.718286 23612 leader_election.cc:341] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e86bf4957fb45abb1ffe19e460f23e4 (127.0.186.195:35205): Not found: Tablet not found: 6c1ec9e57b664ef6a23a1a0979715423
I20260806 22:42:48.718327 23612 leader_election.cc:302] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7871df1715394f92a633b438c5b258de; no voters: 2e41f688ed4d4d768246208cd34e2a49, 5e86bf4957fb45abb1ffe19e460f23e4
I20260806 22:42:48.718395 24300 raft_consensus.cc:2726] T 6c1ec9e57b664ef6a23a1a0979715423 P 7871df1715394f92a633b438c5b258de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:42:49.134436 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.134226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:32773)
0806 22:42:49.134354 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.134386 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260806 22:42:49.199311 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.199062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:48661)
0806 22:42:49.199201 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.199235 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 22:42:49.356813 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:49.356613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:37423)
0806 22:42:49.356730 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:49.356769 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260806 22:42:50.135154 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.134911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:51645)
0806 22:42:50.135066 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.135105 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260806 22:42:50.199965 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.199692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:57907)
0806 22:42:50.199853 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.199922 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260806 22:42:50.357460 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:50.357236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:49497)
0806 22:42:50.357378 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:50.357415 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260806 22:42:51.135826 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.135599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:54343)
0806 22:42:51.135741 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.135779 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260806 22:42:51.200609 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.200406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:41493)
0806 22:42:51.200525 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.200563 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260806 22:42:51.358218 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:51.357963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:57825)
0806 22:42:51.358133 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:51.358174 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260806 22:42:52.136531 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.136284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:46663)
0806 22:42:52.136437 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.136479 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260806 22:42:52.201218 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.201001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:51127)
0806 22:42:52.201143 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.201175 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260806 22:42:52.358853 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:52.358638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:33963)
0806 22:42:52.358773 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:52.358811 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260806 22:42:53.137310 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.137077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.193:50195)
0806 22:42:53.137216 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.137254 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 22:42:53.201953 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.201683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.194:50313)
0806 22:42:53.201817 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.201899 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260806 22:42:53.353271 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 23550
I20260806 22:42:53.364554 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:53.362702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:38545 (local address 127.0.186.195:47229)
0806 22:42:53.364463 (+ 1761us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:53.364501 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":1688,"threads_started":1}
I20260806 22:42:53.374219 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 23808
I20260806 22:42:53.383966 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 24030
2026-08-06T22:42:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18736 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:53 dist-test-slave-gjrz krb5kdc[24324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:53 dist-test-slave-gjrz krb5kdc[24324](info): set up 2 sockets
Aug 06 22:42:53 dist-test-slave-gjrz krb5kdc[24324](info): commencing operation
krb5kdc: starting...
W20260806 22:42:54.506671 747 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:42:54 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:42:54.618919 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:54.697863 24345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:54.698045 24345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:54.698065 24345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:54.699168 24345 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:42:54.699205 24345 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:42:54.699218 24345 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:42:54.699229 24345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:54.699254 24345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:54.699306 24345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:54.699329 24345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:54.699352 24345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:54.700786 24345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:54.701001 24345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:54.701205 24345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:54.705890 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:54.705909 24352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:54.706009 24345 server_base.cc:1034] running on GCE node
W20260806 22:42:54.705906 24357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:54.706271 24345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:54.706532 24345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:54.707666 24345 hybrid_clock.cc:648] HybridClock initialized: now 1786056174707651 us; error 26 us; skew 500 ppm
Aug 06 22:42:54 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:54.709954 24345 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:42:54.710367 24345 webserver.cc:460] Webserver started at http://127.0.186.254:35799/ using document root <none> and password file <none>
I20260806 22:42:54.710554 24345 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:54.710600 24345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:54.710686 24345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:54.711650 24345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "b3b14bfd5bec4cde8beb4a0038c245b2"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "58173da24f46ad5e7f8ba8e6c99eb63f"
server_key_iv: "68cc9c804144be18974dd58531fabb3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.711926 24345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "b3b14bfd5bec4cde8beb4a0038c245b2"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "58173da24f46ad5e7f8ba8e6c99eb63f"
server_key_iv: "68cc9c804144be18974dd58531fabb3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.713050 24345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:54.713662 24368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:54.713822 24345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:54.713901 24345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "b3b14bfd5bec4cde8beb4a0038c245b2"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "58173da24f46ad5e7f8ba8e6c99eb63f"
server_key_iv: "68cc9c804144be18974dd58531fabb3b"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.713955 24345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:54.736742 24345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:54.738349 24345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:54.738453 24345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:54.742161 24345 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37971
I20260806 22:42:54.742220 24447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37971 every 8 connection(s)
I20260806 22:42:54.742512 24345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:54.742936 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 24345
I20260806 22:42:54.743028 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:42:54.743214 747 external_mini_cluster.cc:1428] Setting key 723d1788656c877455a182cce3b49c15
I20260806 22:42:54.743462 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:54.746616 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap starting.
I20260806 22:42:54.747217 24448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:54.747512 24448 log.cc:824] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:54.748121 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: No bootstrap required, opened a new log
Aug 06 22:42:54 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:54.749954 24448 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:42:54.750092 24448 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:54.750151 24448 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Initialized, Role: FOLLOWER
I20260806 22:42:54.750280 24448 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:42:54.750356 24448 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:42:54.750397 24448 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:42:54.750442 24448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:54.750938 24448 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:42:54.751035 24448 leader_election.cc:302] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b14bfd5bec4cde8beb4a0038c245b2; no voters:
I20260806 22:42:54.751209 24448 leader_election.cc:288] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:42:54.751287 24456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:54.751492 24456 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 LEADER]: Becoming Leader. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Running, Role: LEADER
I20260806 22:42:54.751497 24448 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:54.751649 24456 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:42:54.752018 24463 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:42:54.752107 24463 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:54.752331 24468 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b14bfd5bec4cde8beb4a0038c245b2. Latest consensus state: current_term: 1 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:42:54.752700 24468 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
W20260806 22:42:54.753360 24490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:54.753587 24490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:54.753684 24476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:54.753794 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.743741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.0.186.254:37971)
0806 22:42:54.743937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:54.743945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:54.743975 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:42:54.744584 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:54.744594 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:54.744656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:54.744799 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:54.745349 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.746653 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.747397 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.747831 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.747938 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.750074 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:54.750078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:54.750078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:54.750089 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:54.751980 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:54.752408 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:54.752410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:54.752411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:54.752505 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:54.752696 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:54.752696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:54.752697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:54.753016 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:54.753230 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:54.753536 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:42:54.753653 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260806 22:42:54.753849 24476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:54.755496 24476 catalog_manager.cc:1347] Generated new cluster ID: cc2d4eddb11148c8b02469c5184c4a22
I20260806 22:42:54.755550 24476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:54.785904 24476 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:54.786619 24476 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:54.799310 24476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Generated new TSK 0
I20260806 22:42:54.799490 24476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:42:54.831847 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:54.919231 24499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:54.919454 24499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:54.919477 24499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:54.920549 24499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:54.920580 24499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:54.920605 24499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:42:54.922010 24499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:54.922186 24499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:54.922353 24499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:54.926936 24510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:54.926947 24513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:54.926966 24506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:54.927212 24499 server_base.cc:1034] running on GCE node
I20260806 22:42:54.927477 24499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:54.927718 24499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:54.928854 24499 hybrid_clock.cc:648] HybridClock initialized: now 1786056174928840 us; error 30 us; skew 500 ppm
Aug 06 22:42:54 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:54.931221 24499 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:42:54.931628 24499 webserver.cc:460] Webserver started at http://127.0.186.193:33655/ using document root <none> and password file <none>
I20260806 22:42:54.931804 24499 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:54.931843 24499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:54.931926 24499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:54.932857 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "bd3504c82865411c94bb9d2bc2df087a"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "1d16a6e86e4691096f6b4afb2bb2e830"
server_key_iv: "987530a13c243283c753563d7edec84e"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.933151 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "bd3504c82865411c94bb9d2bc2df087a"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "1d16a6e86e4691096f6b4afb2bb2e830"
server_key_iv: "987530a13c243283c753563d7edec84e"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.934302 24499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:54.934892 24524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:54.935045 24499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:54.935125 24499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "bd3504c82865411c94bb9d2bc2df087a"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "1d16a6e86e4691096f6b4afb2bb2e830"
server_key_iv: "987530a13c243283c753563d7edec84e"
server_key_version: "encryptionkey@0"
I20260806 22:42:54.935178 24499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:54.958914 24499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:54.960947 24499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:54.961053 24499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:54.961234 24499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:54.961481 24499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:54.961510 24499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:54.961530 24499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:54.961546 24499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:54.966984 24499 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:39721
I20260806 22:42:54.967074 24696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:39721 every 8 connection(s)
I20260806 22:42:54.967350 24499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:42:54 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:54.976049 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.968045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:35373 (local address 127.0.186.254:37971)
0806 22:42:54.968101 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:54.968104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:54.968282 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:42:54.968806 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:54.968808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:54.968819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:54.968875 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:54.969833 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.970351 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.971334 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.971548 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.971605 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.973608 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:54.973610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:54.973610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:54.973617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:54.974763 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:54.975149 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:54.975151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:54.975151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:54.975203 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:54.975442 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:54.975443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:54.975444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:54.975559 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:54.975616 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:54.975864 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:42:54.975958 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:42:54.976191 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:54.967986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:35373)
0806 22:42:54.968164 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:54.968203 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:54.968698 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:54.968947 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:54.968951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:54.969532 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:54.969722 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.969729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.970444 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.970446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.971222 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:54.971225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:54.971622 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:54.971623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:54.971746 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:54.972166 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:54.972174 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:54.973507 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:54.974855 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:54.974863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:54.974865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:54.975079 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:54.975259 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:54.975280 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:54.975281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:54.975382 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:54.975615 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:54.975623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:54.975748 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:42:54.975836 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:42:54.975988 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260806 22:42:54.976665 24697 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:42:54.976750 24697 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:54.976909 24697 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:42:54.977275 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 24499
I20260806 22:42:54.977376 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:42:54.977386 24394 ts_manager.cc:194] Registered new tserver with Master: bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721)
I20260806 22:42:54.977576 747 external_mini_cluster.cc:1428] Setting key 373c8cc2446cbb23454160d10198c21a
I20260806 22:42:54.978051 24394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:35373
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:42:54.999758 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:55.080614 24704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:55.080806 24704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:55.080828 24704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:55.082027 24704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:55.082072 24704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:55.082096 24704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:42:55.083552 24704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:55.083767 24704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:55.083987 24704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:55.088567 24712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:55.088600 24718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:55.088578 24713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:55.088822 24704 server_base.cc:1034] running on GCE node
I20260806 22:42:55.088989 24704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:55.089186 24704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:55.090315 24704 hybrid_clock.cc:648] HybridClock initialized: now 1786056175090301 us; error 29 us; skew 500 ppm
Aug 06 22:42:55 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:55.092692 24704 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:42:55.093060 24704 webserver.cc:460] Webserver started at http://127.0.186.194:33391/ using document root <none> and password file <none>
I20260806 22:42:55.093251 24704 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:55.093288 24704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:55.093376 24704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:55.094308 24704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "ae98ce2b311740efabc24f724791d417"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "572ac4c8c9b700ea34635f575540eb08"
server_key_iv: "43eb2d41f950940bf9cc16f9901ef42e"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.094588 24704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "ae98ce2b311740efabc24f724791d417"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "572ac4c8c9b700ea34635f575540eb08"
server_key_iv: "43eb2d41f950940bf9cc16f9901ef42e"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.095754 24704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:42:55.096392 24726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.096539 24704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:42:55.096619 24704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "ae98ce2b311740efabc24f724791d417"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "572ac4c8c9b700ea34635f575540eb08"
server_key_iv: "43eb2d41f950940bf9cc16f9901ef42e"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.096678 24704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:55.106444 24704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:55.108522 24704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:55.108654 24704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:55.108857 24704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:55.109146 24704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:55.109180 24704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.109207 24704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:55.109231 24704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.114790 24704 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43381
I20260806 22:42:55.114842 24902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43381 every 8 connection(s)
I20260806 22:42:55.115132 24704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:42:55 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:55.123112 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.115775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:47891 (local address 127.0.186.254:37971)
0806 22:42:55.115823 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.115826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.116016 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:42:55.116471 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.116472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.116482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.116545 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.117414 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.117944 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.118850 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.119029 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.119100 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.120986 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:55.120987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:55.120987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:55.120993 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:55.122119 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.122443 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.122444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.122444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.122488 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.122657 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.122657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.122657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.122750 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:55.122793 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.122982 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.123037 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:55.123297 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.115761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:47891)
0806 22:42:55.115906 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.115939 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.116396 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.116597 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.116601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.117145 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:55.117331 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.117336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.118007 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.118007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.118789 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.118792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.119104 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.119105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.119196 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.119661 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:55.119669 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:55.120914 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:55.122182 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:55.122188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:55.122190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:55.122391 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:55.122530 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:55.122530 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:55.122530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:55.122618 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:55.122789 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:55.122791 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:55.122894 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.122968 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.123084 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260806 22:42:55.123751 24903 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:42:55.123840 24903 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:55.124006 24903 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:42:55.124365 24394 ts_manager.cc:194] Registered new tserver with Master: ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
I20260806 22:42:55.124416 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 24704
I20260806 22:42:55.124509 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:42:55.124730 747 external_mini_cluster.cc:1428] Setting key 7d00eee2e39d2ac01e49757d7f6ac122
I20260806 22:42:55.124841 24394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:47891
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:42:55.148350 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:55.227830 24909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:55.228021 24909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:55.228053 24909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:55.229168 24909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:55.229205 24909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:55.229239 24909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:42:55.230665 24909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:42:55.230896 24909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:55.231102 24909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:55.235656 24920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:55.235664 24925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:55.235740 24923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:55.235832 24909 server_base.cc:1034] running on GCE node
I20260806 22:42:55.236097 24909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:55.236325 24909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:55.237462 24909 hybrid_clock.cc:648] HybridClock initialized: now 1786056175237451 us; error 27 us; skew 500 ppm
Aug 06 22:42:55 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:55.239830 24909 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:42:55.240222 24909 webserver.cc:460] Webserver started at http://127.0.186.195:40489/ using document root <none> and password file <none>
I20260806 22:42:55.240427 24909 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:55.240474 24909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:55.240602 24909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:55.241561 24909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "4984ca7c1fb64c1bb18197932ddc212e"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "614c78c8f06cc8d56dcdc0b88e73a885"
server_key_iv: "26f6b1374aaee3c8639c7b99a0489b48"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.241899 24909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "4984ca7c1fb64c1bb18197932ddc212e"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "614c78c8f06cc8d56dcdc0b88e73a885"
server_key_iv: "26f6b1374aaee3c8639c7b99a0489b48"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.243129 24909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:42:55.243858 24932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.244011 24909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.244093 24909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "4984ca7c1fb64c1bb18197932ddc212e"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "614c78c8f06cc8d56dcdc0b88e73a885"
server_key_iv: "26f6b1374aaee3c8639c7b99a0489b48"
server_key_version: "encryptionkey@0"
I20260806 22:42:55.244161 24909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:55.254007 24909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:55.255712 24909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:55.255839 24909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:55.256024 24909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:42:55.256296 24909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:42:55.256330 24909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.256357 24909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:42:55.256381 24909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:55.262084 24909 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45873
I20260806 22:42:55.262137 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45873 every 8 connection(s)
I20260806 22:42:55.262416 24909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:42:55.262802 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 24909
I20260806 22:42:55.262888 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:42:55.263077 747 external_mini_cluster.cc:1428] Setting key 4b6652e2da46e2ff47e7ea92a45982af
Aug 06 22:42:55 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:42:55.270460 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.263108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:42715 (local address 127.0.186.254:37971)
0806 22:42:55.263159 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.263161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.263441 (+ 280us) server_negotiation.cc:407] Connection header received
0806 22:42:55.263941 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.263942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.263954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.264001 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.264858 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.265368 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.266254 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.266435 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.266477 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.268403 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:55.268404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:55.268405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:55.268411 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:55.269448 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.269789 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.269789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.269790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.269834 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.270011 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.270011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.270012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.270101 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:55.270145 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.270339 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.270398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:42:55.270617 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.263087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:42715)
0806 22:42:55.263258 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.263322 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.263839 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.264054 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.264057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.264612 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:55.264794 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.264798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.265428 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.265428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.266196 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.266198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.266486 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.266486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.266595 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.267044 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:55.267052 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:55.268330 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:55.269510 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:55.269516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:55.269518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:55.269734 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:55.269877 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:55.269877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:55.269878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:55.269971 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:55.270143 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:55.270147 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:55.270250 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.270324 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.270434 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260806 22:42:55.271011 25103 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:42:55.271090 25103 heartbeater.cc:464] Registering TS with master...
I20260806 22:42:55.271251 25103 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:42:55.271615 24394 ts_manager.cc:194] Registered new tserver with Master: 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873)
I20260806 22:42:55.271956 24394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:42715
I20260806 22:42:55.274602 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:42:55.281078 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.275896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.0.186.254:37971)
0806 22:42:55.275942 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.275944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.276044 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:42:55.276117 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.276118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.276127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.276169 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:55.276686 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.277146 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.277828 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.278010 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.278062 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.279053 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:55.279054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:55.279054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:55.279061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:55.280062 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.280363 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.280364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.280364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.280414 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:55.280580 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:55.280581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:55.280582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:55.280715 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:55.280802 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.280923 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.280997 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:42:55.282313 24394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:42:55.283916 24394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:42:55.289966 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.285953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.0.186.193:39721)
0806 22:42:55.286140 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.286145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.286305 (+ 160us) server_negotiation.cc:407] Connection header received
0806 22:42:55.286463 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.286467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.286484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.286570 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:55.286822 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.287587 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.288841 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.289549 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289619 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.289675 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.289834 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.289886 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260806 22:42:55.290024 25124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.286188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.0.186.194:43381)
0806 22:42:55.286355 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.286359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.286415 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:42:55.286497 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.286498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.286511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.286577 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:55.286820 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.287956 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289184 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.289696 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289764 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.289826 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.289908 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.289951 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260806 22:42:55.290024 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.285834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43381 (local address 127.0.0.1:45390)
0806 22:42:55.286357 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.286366 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.286427 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.286667 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.286673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.286676 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:55.286743 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.286745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.288077 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.288078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.289109 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.289111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.289778 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.289844 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.289848 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.289898 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.289956 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":49,"thread_start_us":57,"threads_started":1}
I20260806 22:42:55.290208 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.286030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45873 (local address 127.0.0.1:43200)
0806 22:42:55.286519 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.286527 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.286568 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.286805 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.286805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.286807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:55.286848 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.286849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.288080 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.288081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.289121 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.289124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.290011 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.290012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.290065 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.290068 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.290116 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.290150 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":39,"thread_start_us":41,"threads_started":1}
I20260806 22:42:55.290406 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.286115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.0.186.195:45873)
0806 22:42:55.286285 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.286289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.286619 (+ 330us) server_negotiation.cc:407] Connection header received
0806 22:42:55.286658 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.286660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.286674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.286739 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:55.286921 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.287981 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289209 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.289876 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289930 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.289979 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.290303 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.290344 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260806 22:42:55.290560 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.285813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.0.1:36608)
0806 22:42:55.286202 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.286242 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.286345 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.286665 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.286668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.286670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:55.286737 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.286739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.287679 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.287680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.288759 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.288762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.289629 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.289630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.289705 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.289719 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.290448 (+ 729us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.290495 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":74,"threads_started":1}
I20260806 22:42:55.290814 24805 tablet_service.cc:1467] Processing CreateTablet for tablet 38b3573cba494609ab226a44bb84f66c (DEFAULT_TABLE table=test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:55.291015 25007 tablet_service.cc:1467] Processing CreateTablet for tablet 38b3573cba494609ab226a44bb84f66c (DEFAULT_TABLE table=test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:55.291214 24805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b3573cba494609ab226a44bb84f66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:55.291363 25007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b3573cba494609ab226a44bb84f66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:55.293788 25126 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Bootstrap starting.
I20260806 22:42:55.294101 25125 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Bootstrap starting.
I20260806 22:42:55.294476 25126 tablet_bootstrap.cc:654] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:55.294708 25125 tablet_bootstrap.cc:654] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:55.294596 24589 tablet_service.cc:1467] Processing CreateTablet for tablet 38b3573cba494609ab226a44bb84f66c (DEFAULT_TABLE table=test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:42:55.294930 24589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b3573cba494609ab226a44bb84f66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:55.295035 25126 log.cc:824] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:55.295212 25125 log.cc:824] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:55.295794 25126 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: No bootstrap required, opened a new log
I20260806 22:42:55.295871 25126 ts_tablet_manager.cc:1397] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:55.296016 25125 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: No bootstrap required, opened a new log
I20260806 22:42:55.296084 25125 ts_tablet_manager.cc:1397] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:42:55.297945 25125 raft_consensus.cc:348] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.297945 25126 raft_consensus.cc:348] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.298132 25125 raft_consensus.cc:374] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:55.298132 25126 raft_consensus.cc:374] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:55.298171 25126 raft_consensus.cc:729] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae98ce2b311740efabc24f724791d417, State: Initialized, Role: FOLLOWER
I20260806 22:42:55.298171 25125 raft_consensus.cc:729] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4984ca7c1fb64c1bb18197932ddc212e, State: Initialized, Role: FOLLOWER
I20260806 22:42:55.298293 25125 consensus_queue.cc:260] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.298293 25126 consensus_queue.cc:260] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.298581 25125 ts_tablet_manager.cc:1428] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:42:55.298576 25126 ts_tablet_manager.cc:1428] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:42:55.298607 24903 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
I20260806 22:42:55.298774 25129 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Bootstrap starting.
I20260806 22:42:55.298844 25103 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
I20260806 22:42:55.299444 25129 tablet_bootstrap.cc:654] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:55.300012 25129 log.cc:824] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:55.300693 25129 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: No bootstrap required, opened a new log
I20260806 22:42:55.300748 25129 ts_tablet_manager.cc:1397] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:42:55.302088 25129 raft_consensus.cc:348] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.302228 25129 raft_consensus.cc:374] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:42:55.302249 25129 raft_consensus.cc:729] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3504c82865411c94bb9d2bc2df087a, State: Initialized, Role: FOLLOWER
I20260806 22:42:55.302345 25129 consensus_queue.cc:260] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.302597 25129 ts_tablet_manager.cc:1428] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:42:55.302628 24697 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
W20260806 22:42:55.365769 24904 tablet.cc:2369] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:55.468016 24700 tablet.cc:2369] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:42:55.512948 25106 tablet.cc:2369] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:42:55.584283 25130 raft_consensus.cc:482] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:42:55.584419 25130 raft_consensus.cc:504] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.584811 25130 leader_election.cc:288] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721), 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873)
I20260806 22:42:55.587455 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.584993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.186.194:50229)
0806 22:42:55.585052 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.585069 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.585141 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.585266 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.585267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.585269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:55.585344 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.585346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.585964 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.585965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.586688 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.586690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.587198 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.587199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.587289 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.587292 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.587338 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.587379 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:42:55.587455 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.585097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50229 (local address 127.0.186.193:39721)
0806 22:42:55.585150 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.585152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.585162 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:55.585192 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.585193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.585203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.585252 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:55.585407 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.585888 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.586775 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.587127 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.587188 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.587197 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.587353 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.587389 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:42:55.587772 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.585231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:57363 (local address 127.0.186.195:45873)
0806 22:42:55.585280 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.585282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.585385 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:42:55.585435 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.585436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.585445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.585495 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:42:55.585647 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.586112 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.587004 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.587422 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.587486 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.587495 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.587665 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.587709 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:55.587809 24623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b3573cba494609ab226a44bb84f66c" candidate_uuid: "ae98ce2b311740efabc24f724791d417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3504c82865411c94bb9d2bc2df087a" is_pre_election: true
I20260806 22:42:55.587958 24623 raft_consensus.cc:2454] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae98ce2b311740efabc24f724791d417 in term 0.
I20260806 22:42:55.587983 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.585130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45873 (local address 127.0.186.194:57363)
0806 22:42:55.585312 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.585321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:55.585380 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:55.585505 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:55.585506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:55.585507 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:42:55.585583 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.585585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.586189 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.586190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.586914 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:55.586916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.587496 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.587497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:55.587565 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.587570 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:42:55.587867 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 22:42:55.587911 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260806 22:42:55.588198 24730 leader_election.cc:302] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae98ce2b311740efabc24f724791d417, bd3504c82865411c94bb9d2bc2df087a; no voters:
I20260806 22:42:55.588346 25130 raft_consensus.cc:2781] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:42:55.588305 25030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b3573cba494609ab226a44bb84f66c" candidate_uuid: "ae98ce2b311740efabc24f724791d417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4984ca7c1fb64c1bb18197932ddc212e" is_pre_election: true
I20260806 22:42:55.588389 25130 raft_consensus.cc:482] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:42:55.588410 25130 raft_consensus.cc:3037] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:55.588440 25030 raft_consensus.cc:2454] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae98ce2b311740efabc24f724791d417 in term 0.
I20260806 22:42:55.589170 25130 raft_consensus.cc:504] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.589285 25130 leader_election.cc:288] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [CANDIDATE]: Term 1 election: Requested vote from peers bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721), 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873)
I20260806 22:42:55.589463 25030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b3573cba494609ab226a44bb84f66c" candidate_uuid: "ae98ce2b311740efabc24f724791d417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4984ca7c1fb64c1bb18197932ddc212e"
I20260806 22:42:55.589484 24623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b3573cba494609ab226a44bb84f66c" candidate_uuid: "ae98ce2b311740efabc24f724791d417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3504c82865411c94bb9d2bc2df087a"
I20260806 22:42:55.589534 25030 raft_consensus.cc:3037] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:55.589573 24623 raft_consensus.cc:3037] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:55.590232 25030 raft_consensus.cc:2454] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae98ce2b311740efabc24f724791d417 in term 1.
I20260806 22:42:55.590308 24623 raft_consensus.cc:2454] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae98ce2b311740efabc24f724791d417 in term 1.
I20260806 22:42:55.590389 24730 leader_election.cc:302] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417; no voters:
I20260806 22:42:55.590464 25130 raft_consensus.cc:2781] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:55.590636 25130 raft_consensus.cc:686] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 1 LEADER]: Becoming Leader. State: Replica: ae98ce2b311740efabc24f724791d417, State: Running, Role: LEADER
I20260806 22:42:55.590739 25130 consensus_queue.cc:237] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:42:55.591563 24394 catalog_manager.cc:5495] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae98ce2b311740efabc24f724791d417 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "ae98ce2b311740efabc24f724791d417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } health_report { overall_health: HEALTHY } } }
I20260806 22:42:55.640362 25124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.638023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.0.186.194:43381)
0806 22:42:55.638095 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:55.638097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:55.638105 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:55.638145 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:55.638146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:55.638164 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:55.638225 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:55.638364 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.638857 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.639635 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:55.639825 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:55.639881 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:55.640034 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:55.640150 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:55.640223 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:55.640260 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:42:55.640280 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:42:55.642028 25030 raft_consensus.cc:1261] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Refusing update from remote peer ae98ce2b311740efabc24f724791d417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:55.642060 24623 raft_consensus.cc:1261] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Refusing update from remote peer ae98ce2b311740efabc24f724791d417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:55.642364 25130 consensus_queue.cc:1035] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:55.642516 25141 consensus_queue.cc:1035] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:55.644995 25146 mvcc.cc:205] Tried to move back new op lower bound from 7315686095427301376 to 7315686095220191232. Current Snapshot: MvccSnapshot[applied={T|T < 7315686095427301376}]
I20260806 22:42:55.647522 24394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:55.647631 24394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:42:55.648305 24394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Sending DeleteTablet for 3 replicas of tablet 38b3573cba494609ab226a44bb84f66c
I20260806 22:42:55.648623 24805 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b3573cba494609ab226a44bb84f66c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260806 22:42:55.648658 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 24499
I20260806 22:42:55.648659 24589 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b3573cba494609ab226a44bb84f66c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:36608
I20260806 22:42:55.648895 25007 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b3573cba494609ab226a44bb84f66c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:43200
I20260806 22:42:55.649094 25159 tablet_replica.cc:330] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: stopping tablet replica
I20260806 22:42:55.649188 25159 raft_consensus.cc:2229] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:42:55.649295 25159 raft_consensus.cc:2258] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:55.649436 25160 tablet_replica.cc:330] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: stopping tablet replica
I20260806 22:42:55.649513 25160 raft_consensus.cc:2229] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:42:55.649596 25160 raft_consensus.cc:2258] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:42:55.652706 25160 ts_tablet_manager.cc:1905] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:55.653777 25159 ts_tablet_manager.cc:1905] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:42:55.654413 25160 ts_tablet_manager.cc:1918] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:55.654477 25160 log.cc:1196] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/38b3573cba494609ab226a44bb84f66c
I20260806 22:42:55.654749 25160 ts_tablet_manager.cc:1939] T 38b3573cba494609ab226a44bb84f66c P 4984ca7c1fb64c1bb18197932ddc212e: Deleting consensus metadata
I20260806 22:42:55.655148 24374 catalog_manager.cc:4841] TS 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): tablet 38b3573cba494609ab226a44bb84f66c (table test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]) successfully deleted
I20260806 22:42:55.655891 25159 ts_tablet_manager.cc:1918] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:42:55.655993 25159 log.cc:1196] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/38b3573cba494609ab226a44bb84f66c
I20260806 22:42:55.656242 25159 ts_tablet_manager.cc:1939] T 38b3573cba494609ab226a44bb84f66c P ae98ce2b311740efabc24f724791d417: Deleting consensus metadata
I20260806 22:42:55.657150 24370 catalog_manager.cc:4841] TS ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): tablet 38b3573cba494609ab226a44bb84f66c (table test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]) successfully deleted
W20260806 22:42:55.662282 24374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:39721 (error 108)
I20260806 22:42:55.662729 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:55.662616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.0.1:36624)
0806 22:42:55.662666 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:55.662700 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:42:55.662858 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 24704
W20260806 22:42:55.662858 24374 catalog_manager.cc:4584] TS bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38b3573cba494609ab226a44bb84f66c: Network error: Client connection negotiation failed: client connection to 127.0.186.193:39721: connect: Connection refused (error 111)
I20260806 22:42:55.670837 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 24909
I20260806 22:42:55.679811 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 24345
I20260806 22:43:15.688755 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--webserver_interface=127.0.186.254
--webserver_port=35799
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:15.768702 25166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:15.768899 25166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:15.768927 25166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:15.770042 25166 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:43:15.770082 25166 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:43:15.770099 25166 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:43:15.770114 25166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:15.770133 25166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:15.770149 25166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:15.770171 25166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:15.770195 25166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:15.771703 25166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=35799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:15.771943 25166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:15.772156 25166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:15.781121 25178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:15.781147 25176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:15.781148 25166 server_base.cc:1034] running on GCE node
W20260806 22:43:15.781137 25174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:15.781641 25166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:15.781948 25166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:15.783074 25166 hybrid_clock.cc:648] HybridClock initialized: now 1786056195783058 us; error 37 us; skew 500 ppm
Aug 06 22:43:15 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:15.785502 25166 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:43:15.785943 25166 webserver.cc:460] Webserver started at http://127.0.186.254:35799/ using document root <none> and password file <none>
I20260806 22:43:15.786150 25166 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:15.786207 25166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:15.787528 25166 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:15.788122 25188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:15.788261 25166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:15.788341 25166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "b3b14bfd5bec4cde8beb4a0038c245b2"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "58173da24f46ad5e7f8ba8e6c99eb63f"
server_key_iv: "68cc9c804144be18974dd58531fabb3b"
server_key_version: "encryptionkey@0"
I20260806 22:43:15.788725 25166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:15.798909 25166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:15.800712 25166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:15.800825 25166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:15.804834 25166 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37971
I20260806 22:43:15.804879 25266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37971 every 8 connection(s)
I20260806 22:43:15.805226 25166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:15.806660 25267 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:43:15.807386 25267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap starting.
I20260806 22:43:15.814095 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 25166
I20260806 22:43:15.814363 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:39721
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=33655
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:43:15.816491 25267 log.cc:824] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:15.818282 25267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:43:15.818555 25267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap complete.
I20260806 22:43:15.820721 25267 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:15.820940 25267 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Initialized, Role: FOLLOWER
I20260806 22:43:15.821086 25267 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:15.821161 25267 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:15.821218 25267 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:15.821265 25267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:43:15.822258 25267 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:15.822357 25267 leader_election.cc:302] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b14bfd5bec4cde8beb4a0038c245b2; no voters:
I20260806 22:43:15.822513 25267 leader_election.cc:288] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:43:15.822578 25274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:43:15.822772 25267 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:15.822788 25274 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 LEADER]: Becoming Leader. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Running, Role: LEADER
I20260806 22:43:15.822881 25274 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:15.823236 25274 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b14bfd5bec4cde8beb4a0038c245b2. Latest consensus state: current_term: 2 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:43:15.823333 25274 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:15.823735 25275 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:43:15.823807 25275 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:15.823930 25289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:15.824712 25289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:15.824810 25289 catalog_manager.cc:1259] Loaded cluster ID: cc2d4eddb11148c8b02469c5184c4a22
I20260806 22:43:15.824839 25289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:15.825986 25289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:15.826164 25289 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Loaded TSK: 0
I20260806 22:43:15.837237 25289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Generated new TSK 1
I20260806 22:43:15.837327 25289 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:43:15.901067 25270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:15.901255 25270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:15.901278 25270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:15.902540 25270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:15.902592 25270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:15.902618 25270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:15.904138 25270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:39721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=33655
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:15.904346 25270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:15.904556 25270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:15.909266 25311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:15.909273 25310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:15.909286 25313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:15.909742 25270 server_base.cc:1034] running on GCE node
I20260806 22:43:15.909955 25270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:15.910218 25270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:15.911336 25270 hybrid_clock.cc:648] HybridClock initialized: now 1786056195911324 us; error 30 us; skew 500 ppm
Aug 06 22:43:15 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:15.914242 25270 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:43:15.914781 25270 webserver.cc:460] Webserver started at http://127.0.186.193:33655/ using document root <none> and password file <none>
I20260806 22:43:15.915025 25270 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:15.915114 25270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:15.916764 25270 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:15.917469 25323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:15.917640 25270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:15.917725 25270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "bd3504c82865411c94bb9d2bc2df087a"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "1d16a6e86e4691096f6b4afb2bb2e830"
server_key_iv: "987530a13c243283c753563d7edec84e"
server_key_version: "encryptionkey@0"
I20260806 22:43:15.918043 25270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:15.930184 25270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:15.932164 25270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:15.932281 25270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:15.932484 25270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:15.932878 25335 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:43:15.934695 25270 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:43:15.934736 25270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:43:15.934767 25270 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:43:15.935411 25270 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:43:15.935444 25270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:15.935529 25335 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Bootstrap starting.
I20260806 22:43:15.942096 25270 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:39721
I20260806 22:43:15.942435 25270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:15.943596 25505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:39721 every 8 connection(s)
I20260806 22:43:15.946853 25335 log.cc:824] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:43:15 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:15.949695 25335 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:43:15.949941 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 25270
I20260806 22:43:15.950033 25335 tablet_bootstrap.cc:492] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Bootstrap complete.
I20260806 22:43:15.950507 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:43381
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=33391
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:43:15.951251 25335 ts_tablet_manager.cc:1397] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.011s
I20260806 22:43:15.952513 25335 raft_consensus.cc:348] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:43:15.952809 25335 raft_consensus.cc:729] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3504c82865411c94bb9d2bc2df087a, State: Initialized, Role: FOLLOWER
I20260806 22:43:15.952970 25335 consensus_queue.cc:260] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } } peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } }
I20260806 22:43:15.953027 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:15.943028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:37317)
0806 22:43:15.943228 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:15.943304 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:15.943879 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:15.944282 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:15.944285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:15.944548 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:15.944723 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:15.944728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.945850 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.945851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:15.946623 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:15.946626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.947360 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.947362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:15.947507 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:15.947999 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:15.948008 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:15.949302 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:15.951374 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:15.951381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:15.951383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:15.951648 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:15.951924 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:15.951925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:15.951926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:15.952045 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:15.952513 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:15.952521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:15.952633 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 22:43:15.952694 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:43:15.952868 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260806 22:43:15.953215 25335 ts_tablet_manager.cc:1428] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:15.953591 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:15.943132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:37317 (local address 127.0.186.254:37971)
0806 22:43:15.943388 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:15.943396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:15.943436 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:43:15.944087 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:15.944092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:15.944135 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:15.944245 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:15.944836 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.945728 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.946733 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.947243 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.947374 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:15.949415 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:15.949419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:15.949419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:15.949428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:15.951258 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:15.951740 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:15.951743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:15.951744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:15.951855 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:15.952127 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:15.952128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:15.952128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:15.952435 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:15.953189 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:15.953273 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:43:15.953411 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260806 22:43:15.954154 25506 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:43:15.954239 25506 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:15.954396 25506 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:43:15.955027 25218 ts_manager.cc:194] Registered new tserver with Master: bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721)
I20260806 22:43:15.956051 25218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:37317
I20260806 22:43:15.956925 25506 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
I20260806 22:43:15.962118 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:15.956305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.0.186.193:39721)
0806 22:43:15.956435 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:15.956439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:15.959442 (+ 3003us) server_negotiation.cc:407] Connection header received
0806 22:43:15.959519 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:15.959521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:15.959535 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:15.959611 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:15.959725 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.960362 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.961286 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.961758 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.961820 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:15.961885 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:15.962022 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:43:15.962060 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260806 22:43:15.962126 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:15.955601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.0.1:45308)
0806 22:43:15.959355 (+ 3754us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:15.959383 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:15.959447 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:15.959615 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:15.959618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:15.959619 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:15.959675 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:15.959677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.960436 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.960437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:15.961209 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:15.961211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:15.961832 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:15.961833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:15.961926 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:15.961940 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:43:15.962002 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:43:15.962052 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3727,"thread_start_us":77,"threads_started":1}
I20260806 22:43:15.962617 25405 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b3573cba494609ab226a44bb84f66c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:42:55 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:45308
I20260806 22:43:15.963018 25518 tablet_replica.cc:330] stopping tablet replica
I20260806 22:43:15.963120 25518 raft_consensus.cc:2229] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:15.963187 25518 raft_consensus.cc:2258] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:15.963475 25518 ts_tablet_manager.cc:1905] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:15.965358 25518 ts_tablet_manager.cc:1918] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:15.965425 25518 log.cc:1196] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal/wals/38b3573cba494609ab226a44bb84f66c
I20260806 22:43:15.965608 25518 ts_tablet_manager.cc:1939] T 38b3573cba494609ab226a44bb84f66c P bd3504c82865411c94bb9d2bc2df087a: Deleting consensus metadata
I20260806 22:43:15.966032 25192 catalog_manager.cc:4841] TS bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721): tablet 38b3573cba494609ab226a44bb84f66c (table test-table [id=3719c61fa9ca4ee1a9a536a1ffac50bd]) successfully deleted
W20260806 22:43:16.035573 25512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:16.035763 25512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:16.035786 25512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:16.036929 25512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:16.036969 25512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:16.036998 25512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:16.038422 25512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:43381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=33391
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:16.038643 25512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:16.038843 25512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:16.043628 25525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:16.043664 25526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:16.043658 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:16.043720 25512 server_base.cc:1034] running on GCE node
I20260806 22:43:16.044001 25512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:16.044237 25512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:16.045387 25512 hybrid_clock.cc:648] HybridClock initialized: now 1786056196045367 us; error 27 us; skew 500 ppm
Aug 06 22:43:16 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:16.047768 25512 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:43:16.048166 25512 webserver.cc:460] Webserver started at http://127.0.186.194:33391/ using document root <none> and password file <none>
I20260806 22:43:16.048358 25512 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:16.048418 25512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:16.049664 25512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:16.050199 25535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.050359 25512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.050441 25512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "ae98ce2b311740efabc24f724791d417"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "572ac4c8c9b700ea34635f575540eb08"
server_key_iv: "43eb2d41f950940bf9cc16f9901ef42e"
server_key_version: "encryptionkey@0"
I20260806 22:43:16.050762 25512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:16.069936 25512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:16.071979 25512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:16.072127 25512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:16.072357 25512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:16.072660 25512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:16.072695 25512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.072723 25512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:16.072741 25512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.078383 25512 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43381
I20260806 22:43:16.078419 25720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43381 every 8 connection(s)
I20260806 22:43:16.078786 25512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:43:16 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:16.087368 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.079584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:43241 (local address 127.0.186.254:37971)
0806 22:43:16.079651 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.079653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.079838 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:43:16.080314 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.080315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.080331 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.080380 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:16.081285 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.081768 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.082693 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.082880 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.082927 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.084935 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:16.084937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:16.084937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:16.084945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:16.086127 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.086515 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.086515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.086516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.086569 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.086763 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.086764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.086764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.086871 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:16.086920 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.087187 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.087282 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:43:16.087512 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.079525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:43241)
0806 22:43:16.079723 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.079761 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.080235 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.080435 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.080439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.081024 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:16.081218 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.081223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.081829 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.081830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.082630 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.082632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.082972 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.082974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.083097 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.083520 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:16.083528 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:16.084830 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:16.086213 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:16.086221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:16.086222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:16.086437 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:16.086617 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:16.086618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:16.086618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:16.086717 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:16.086947 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:16.086953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:16.087077 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.087166 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.087329 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260806 22:43:16.087932 25722 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:43:16.088014 25722 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:16.088179 25722 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:43:16.088296 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 25512
I20260806 22:43:16.088569 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:45873
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=40489
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:43:16.088737 25218 ts_manager.cc:194] Registered new tserver with Master: ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
I20260806 22:43:16.089241 25218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:43241
W20260806 22:43:16.168876 25727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:16.169068 25727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:16.169099 25727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:16.170218 25727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:16.170259 25727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:16.170293 25727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:16.171799 25727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:45873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=40489
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:16.172027 25727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:16.172233 25727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:16.176985 25734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:16.177068 25733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:16.177243 25727 server_base.cc:1034] running on GCE node
W20260806 22:43:16.177289 25738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:16.177484 25727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:16.177744 25727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:16.178889 25727 hybrid_clock.cc:648] HybridClock initialized: now 1786056196178863 us; error 33 us; skew 500 ppm
Aug 06 22:43:16 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:16.181268 25727 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:43:16.181707 25727 webserver.cc:460] Webserver started at http://127.0.186.195:40489/ using document root <none> and password file <none>
I20260806 22:43:16.181913 25727 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:16.181978 25727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:16.183208 25727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:16.183769 25747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.183907 25727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.183988 25727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "4984ca7c1fb64c1bb18197932ddc212e"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "614c78c8f06cc8d56dcdc0b88e73a885"
server_key_iv: "26f6b1374aaee3c8639c7b99a0489b48"
server_key_version: "encryptionkey@0"
I20260806 22:43:16.184314 25727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:16.193817 25727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:16.195576 25727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:16.195703 25727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:16.195899 25727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:16.196172 25727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:16.196205 25727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.196233 25727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:16.196250 25727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.201683 25727 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45873
I20260806 22:43:16.201740 25937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45873 every 8 connection(s)
I20260806 22:43:16.202061 25727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:16.203934 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 25727
Aug 06 22:43:16 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:16.210821 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.204335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.0.186.254:37971)
0806 22:43:16.204447 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.204448 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.204532 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:43:16.204581 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.204582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.204594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.204650 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:16.205371 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.206099 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.206940 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.207239 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.207332 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.208359 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:16.208361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:16.208361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:16.208368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:16.209791 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.210133 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.210134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.210135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.210195 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.210356 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.210357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.210358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.210485 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:16.210663 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.210695 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.210755 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260806 22:43:16.211838 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.202652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:50391 (local address 127.0.186.254:37971)
0806 22:43:16.202715 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.202717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.202903 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:43:16.203412 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.203413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.203426 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.203473 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:16.204587 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.205358 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.206581 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.206876 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.206937 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.209174 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:16.209176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:16.209176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:16.209183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:16.210538 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.210964 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.210965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.210965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.211019 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.211281 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.211282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.211283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.211402 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:16.211461 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.211720 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.211782 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":37}
I20260806 22:43:16.211983 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.202629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:50391)
0806 22:43:16.202792 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.202830 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.203318 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.203553 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.203557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.204237 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:16.204487 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.204494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.205438 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.205439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.206479 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.206482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.206953 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.206954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.207087 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.207604 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:16.207612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:16.209068 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:16.210611 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:16.210619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:16.210621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:16.210901 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:16.211073 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:16.211074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:16.211074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:16.211204 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:16.211468 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:16.211476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:16.211614 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.211696 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.211810 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260806 22:43:16.212356 25941 heartbeater.cc:347] Connected to a master server at 127.0.186.254:37971
I20260806 22:43:16.212445 25941 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:16.212617 25941 heartbeater.cc:511] Master 127.0.186.254:37971 requested a full tablet report, sending...
I20260806 22:43:16.212951 25218 ts_manager.cc:194] Registered new tserver with Master: 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873)
I20260806 22:43:16.213282 25218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:50391
I20260806 22:43:16.222262 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:16.224852 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.222568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.0.186.254:37971)
0806 22:43:16.222615 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.222617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.222704 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:43:16.222763 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.222763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.222794 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.222848 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:16.222982 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.223541 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.224306 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.224478 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.224519 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.224668 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:16.224705 (+ 37us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:43:16.224808 (+ 103us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46606: token expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:43:16.224896 25509 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46606: token expired
W20260806 22:43:16.225061 25108 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.186.254:37971)
I20260806 22:43:16.229794 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.225300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.0.186.254:37971)
0806 22:43:16.225330 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.225332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.225340 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:43:16.225376 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.225377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.225383 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.225425 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:16.225824 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.226248 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.226829 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.226986 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.227023 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.227971 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:16.227972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:16.227972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:16.227978 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:16.229007 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.229290 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.229291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.229291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.229335 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:16.229439 (+ 104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:16.229440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:16.229440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:16.229546 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:16.229616 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.229687 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.229741 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 22:43:16.230760 25218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:16.232316 25218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:16.234357 25405 tablet_service.cc:1467] Processing CreateTablet for tablet 088ef692af9246dcbaf51ff13624ccfa (DEFAULT_TABLE table=test-table [id=7a0145dd9fe047548f7ef3e2e54f0741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:16.234563 25405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088ef692af9246dcbaf51ff13624ccfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:16.236078 25335 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Bootstrap starting.
I20260806 22:43:16.236366 25335 tablet_bootstrap.cc:654] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:16.236845 25335 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: No bootstrap required, opened a new log
I20260806 22:43:16.236894 25335 ts_tablet_manager.cc:1397] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:16.236996 25335 raft_consensus.cc:348] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.237058 25335 raft_consensus.cc:374] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:16.237105 25335 raft_consensus.cc:729] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3504c82865411c94bb9d2bc2df087a, State: Initialized, Role: FOLLOWER
I20260806 22:43:16.237156 25335 consensus_queue.cc:260] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.237241 25335 ts_tablet_manager.cc:1428] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:16.237812 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.233906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45873 (local address 127.0.0.1:58042)
0806 22:43:16.234019 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.234028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.234059 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.234349 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.234349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.234351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:16.234388 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.234389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.235682 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.235683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.236805 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.236808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.237585 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.237586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.237647 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.237651 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.237697 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.237741 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":45}
I20260806 22:43:16.237936 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.233907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43381 (local address 127.0.0.1:49510)
0806 22:43:16.234306 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.234315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.234364 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.234547 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.234548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.234550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:16.234591 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.234592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.235631 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.235632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.236821 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.236823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.237696 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.237697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.237762 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.237766 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.237822 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.237865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":48,"thread_start_us":84,"threads_started":1}
I20260806 22:43:16.237965 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.234104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.0.186.194:43381)
0806 22:43:16.234279 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.234283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.234414 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:43:16.234457 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.234459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.234474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.234545 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:16.234639 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.235555 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.236895 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.237626 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.237695 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.237754 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.237840 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.237885 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260806 22:43:16.238302 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.234014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.0.186.195:45873)
0806 22:43:16.234191 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.234196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.234208 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:43:16.234248 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.234249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.234264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.234348 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:16.234438 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.235611 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.236903 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.237516 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.237579 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.237640 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.238187 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.238231 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260806 22:43:16.238772 25614 tablet_service.cc:1467] Processing CreateTablet for tablet 088ef692af9246dcbaf51ff13624ccfa (DEFAULT_TABLE table=test-table [id=7a0145dd9fe047548f7ef3e2e54f0741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:16.238924 25824 tablet_service.cc:1467] Processing CreateTablet for tablet 088ef692af9246dcbaf51ff13624ccfa (DEFAULT_TABLE table=test-table [id=7a0145dd9fe047548f7ef3e2e54f0741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:16.239158 25614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088ef692af9246dcbaf51ff13624ccfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:16.239215 25824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088ef692af9246dcbaf51ff13624ccfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:16.242637 25959 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Bootstrap starting.
I20260806 22:43:16.242920 25960 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Bootstrap starting.
I20260806 22:43:16.243594 25960 tablet_bootstrap.cc:654] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:16.243937 25959 tablet_bootstrap.cc:654] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:16.244200 25960 log.cc:824] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:16.244535 25959 log.cc:824] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:16.244976 25960 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: No bootstrap required, opened a new log
I20260806 22:43:16.245043 25960 ts_tablet_manager.cc:1397] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:16.245208 25959 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: No bootstrap required, opened a new log
I20260806 22:43:16.245263 25959 ts_tablet_manager.cc:1397] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:43:16.246574 25960 raft_consensus.cc:348] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.246630 25959 raft_consensus.cc:348] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.246721 25960 raft_consensus.cc:374] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:16.246749 25959 raft_consensus.cc:374] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:16.246754 25960 raft_consensus.cc:729] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae98ce2b311740efabc24f724791d417, State: Initialized, Role: FOLLOWER
I20260806 22:43:16.246789 25959 raft_consensus.cc:729] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4984ca7c1fb64c1bb18197932ddc212e, State: Initialized, Role: FOLLOWER
I20260806 22:43:16.246867 25960 consensus_queue.cc:260] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.246945 25959 consensus_queue.cc:260] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.247167 25960 ts_tablet_manager.cc:1428] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:16.247212 25722 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
I20260806 22:43:16.247236 25959 ts_tablet_manager.cc:1428] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:16.247290 25941 heartbeater.cc:503] Master 127.0.186.254:37971 was elected leader, sending a full tablet report...
I20260806 22:43:16.306414 25965 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:16.306536 25965 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.306928 25965 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381), bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721)
I20260806 22:43:16.309782 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.307168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:57541 (local address 127.0.186.194:43381)
0806 22:43:16.307207 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.307209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.307216 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:43:16.307247 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.307248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.307257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.307346 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:16.307503 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.308021 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.308933 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.309394 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.309493 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.309504 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.309682 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.309717 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:43:16.309784 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.307155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:58987 (local address 127.0.186.193:39721)
0806 22:43:16.307200 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.307202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.307370 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:43:16.307423 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.307424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.307435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.307493 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:16.307632 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.308103 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.308991 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.309492 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.309557 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.309565 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.309682 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.309719 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:43:16.309794 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.307071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43381 (local address 127.0.186.195:57541)
0806 22:43:16.307129 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.307139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.307179 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.307352 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.307354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.307356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:16.307441 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.307443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.308097 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.308098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.308849 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.308851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.309519 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.309520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.309618 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.309621 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.309669 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.309710 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260806 22:43:16.309798 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.307109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.186.195:58987)
0806 22:43:16.307299 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.307310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:16.307366 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:16.307501 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:16.307502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:16.307503 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:16.307570 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.307572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.308175 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.308176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.308907 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:16.308910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.309558 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.309558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:16.309617 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.309620 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:16.309665 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:43:16.309704 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260806 22:43:16.310235 25649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088ef692af9246dcbaf51ff13624ccfa" candidate_uuid: "4984ca7c1fb64c1bb18197932ddc212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae98ce2b311740efabc24f724791d417" is_pre_election: true
I20260806 22:43:16.310375 25649 raft_consensus.cc:2454] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4984ca7c1fb64c1bb18197932ddc212e in term 0.
I20260806 22:43:16.310356 25433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088ef692af9246dcbaf51ff13624ccfa" candidate_uuid: "4984ca7c1fb64c1bb18197932ddc212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3504c82865411c94bb9d2bc2df087a" is_pre_election: true
I20260806 22:43:16.310520 25433 raft_consensus.cc:2454] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4984ca7c1fb64c1bb18197932ddc212e in term 0.
I20260806 22:43:16.310590 25749 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417; no voters:
I20260806 22:43:16.310705 25965 raft_consensus.cc:2781] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:16.310755 25965 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:16.310778 25965 raft_consensus.cc:3037] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:16.311494 25965 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.311626 25965 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [CANDIDATE]: Term 1 election: Requested vote from peers ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381), bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721)
I20260806 22:43:16.311794 25649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088ef692af9246dcbaf51ff13624ccfa" candidate_uuid: "4984ca7c1fb64c1bb18197932ddc212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae98ce2b311740efabc24f724791d417"
I20260806 22:43:16.311873 25649 raft_consensus.cc:3037] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:16.312559 25649 raft_consensus.cc:2454] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4984ca7c1fb64c1bb18197932ddc212e in term 1.
I20260806 22:43:16.312669 25433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088ef692af9246dcbaf51ff13624ccfa" candidate_uuid: "4984ca7c1fb64c1bb18197932ddc212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3504c82865411c94bb9d2bc2df087a"
I20260806 22:43:16.312738 25749 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417; no voters:
I20260806 22:43:16.312757 25433 raft_consensus.cc:3037] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:16.312826 25965 raft_consensus.cc:2781] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:16.312996 25965 raft_consensus.cc:686] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 1 LEADER]: Becoming Leader. State: Replica: 4984ca7c1fb64c1bb18197932ddc212e, State: Running, Role: LEADER
I20260806 22:43:16.313071 25965 consensus_queue.cc:237] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:16.313467 25433 raft_consensus.cc:2454] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4984ca7c1fb64c1bb18197932ddc212e in term 1.
I20260806 22:43:16.313871 25217 catalog_manager.cc:5495] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195). New cstate: current_term: 1 leader_uuid: "4984ca7c1fb64c1bb18197932ddc212e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } health_report { overall_health: UNKNOWN } } }
W20260806 22:43:16.329296 25724 tablet.cc:2369] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:16.331913 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.329798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.0.186.195:45873)
0806 22:43:16.329839 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:16.329841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:16.329852 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:16.329915 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:16.329916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:16.329933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:16.329989 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:16.330077 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.330526 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.331192 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:16.331441 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:16.331522 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:16.331635 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:16.331736 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:16.331793 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:16.331805 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 22:43:16.331822 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:43:16.333674 25649 raft_consensus.cc:1261] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Refusing update from remote peer 4984ca7c1fb64c1bb18197932ddc212e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:16.333712 25433 raft_consensus.cc:1261] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Refusing update from remote peer 4984ca7c1fb64c1bb18197932ddc212e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:16.334007 25968 consensus_queue.cc:1035] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:16.334136 25965 consensus_queue.cc:1035] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:16.336292 25973 mvcc.cc:205] Tried to move back new op lower bound from 7315686180180172800 to 7315686180099002368. Current Snapshot: MvccSnapshot[applied={T|T < 7315686180180172800}]
I20260806 22:43:16.336498 25978 mvcc.cc:205] Tried to move back new op lower bound from 7315686180180172800 to 7315686180099002368. Current Snapshot: MvccSnapshot[applied={T|T < 7315686180180172800}]
I20260806 22:43:16.339030 25218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:16.339128 25218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:16.339826 25218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Sending DeleteTablet for 3 replicas of tablet 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:16.340118 25824 tablet_service.cc:1514] Processing DeleteTablet for tablet 088ef692af9246dcbaf51ff13624ccfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:16 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:58042
I20260806 22:43:16.340126 25614 tablet_service.cc:1514] Processing DeleteTablet for tablet 088ef692af9246dcbaf51ff13624ccfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:16 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:49510
I20260806 22:43:16.340345 25405 tablet_service.cc:1514] Processing DeleteTablet for tablet 088ef692af9246dcbaf51ff13624ccfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:16 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:45308
I20260806 22:43:16.340354 25992 tablet_replica.cc:330] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: stopping tablet replica
I20260806 22:43:16.340430 25518 tablet_replica.cc:330] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: stopping tablet replica
I20260806 22:43:16.340456 25992 raft_consensus.cc:2229] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:16.340479 25518 raft_consensus.cc:2229] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:16.340530 25518 raft_consensus.cc:2258] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:16.340565 25992 raft_consensus.cc:2258] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:16.340830 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 25270
I20260806 22:43:16.340843 25990 tablet_replica.cc:330] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: stopping tablet replica
I20260806 22:43:16.340858 25518 ts_tablet_manager.cc:1905] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:16.346974 25990 raft_consensus.cc:2229] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:43:16.347155 25990 raft_consensus.cc:2258] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:16.348845 25990 ts_tablet_manager.cc:1905] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:16.349350 25992 ts_tablet_manager.cc:1905] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:16.350715 25990 ts_tablet_manager.cc:1918] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:16.350844 25990 log.cc:1196] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:16.351089 25992 ts_tablet_manager.cc:1918] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:16.351229 25992 log.cc:1196] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:16.351248 25990 ts_tablet_manager.cc:1939] T 088ef692af9246dcbaf51ff13624ccfa P 4984ca7c1fb64c1bb18197932ddc212e: Deleting consensus metadata
I20260806 22:43:16.351576 25992 ts_tablet_manager.cc:1939] T 088ef692af9246dcbaf51ff13624ccfa P ae98ce2b311740efabc24f724791d417: Deleting consensus metadata
I20260806 22:43:16.352044 25189 catalog_manager.cc:4841] TS ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): tablet 088ef692af9246dcbaf51ff13624ccfa (table test-table [id=7a0145dd9fe047548f7ef3e2e54f0741]) successfully deleted
W20260806 22:43:16.355062 25192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:39721 (error 108)
I20260806 22:43:16.355473 25192 catalog_manager.cc:4841] TS 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): tablet 088ef692af9246dcbaf51ff13624ccfa (table test-table [id=7a0145dd9fe047548f7ef3e2e54f0741]) successfully deleted
I20260806 22:43:16.355556 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:16.355427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:39721 (local address 127.0.0.1:45312)
0806 22:43:16.355477 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:16.355518 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:43:16.355571 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 25512
W20260806 22:43:16.357283 25192 catalog_manager.cc:4584] TS bd3504c82865411c94bb9d2bc2df087a (127.0.186.193:39721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 088ef692af9246dcbaf51ff13624ccfa: Network error: Client connection negotiation failed: client connection to 127.0.186.193:39721: connect: Connection refused (error 111)
I20260806 22:43:16.362641 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 25727
I20260806 22:43:16.373998 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 25166
I20260806 22:43:36.384187 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--webserver_interface=127.0.186.254
--webserver_port=35799
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:36.465508 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:36.465713 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:36.465744 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:36.466846 25996 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:43:36.466884 25996 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:43:36.466900 25996 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:43:36.466914 25996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:36.466938 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:36.466957 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:36.466979 25996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:36.466996 25996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:36.468498 25996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=35799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:36.468743 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:36.468946 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:36.473454 26009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.473410 26005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.473404 26004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:36.473770 25996 server_base.cc:1034] running on GCE node
I20260806 22:43:36.474184 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:36.474429 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:36.475565 25996 hybrid_clock.cc:648] HybridClock initialized: now 1786056216475553 us; error 26 us; skew 500 ppm
Aug 06 22:43:36 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:36.477897 25996 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:43:36.478305 25996 webserver.cc:460] Webserver started at http://127.0.186.254:35799/ using document root <none> and password file <none>
I20260806 22:43:36.478507 25996 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:36.478547 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:36.479815 25996 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:36.480394 26022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.480536 25996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:36.480618 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "b3b14bfd5bec4cde8beb4a0038c245b2"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "58173da24f46ad5e7f8ba8e6c99eb63f"
server_key_iv: "68cc9c804144be18974dd58531fabb3b"
server_key_version: "encryptionkey@0"
I20260806 22:43:36.480924 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:36.492425 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:36.494117 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:36.494234 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:36.498176 25996 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37971
I20260806 22:43:36.498204 26103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37971 every 8 connection(s)
I20260806 22:43:36.498558 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:36.499662 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 25996
I20260806 22:43:36.499935 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:39721
--local_ip_for_outbound_sockets=127.0.186.193
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=33655
--webserver_interface=127.0.186.193
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:43:36.500114 26104 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:43:36.500768 26104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap starting.
I20260806 22:43:36.509682 26104 log.cc:824] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:36.511901 26104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:43:36.512166 26104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Bootstrap complete.
I20260806 22:43:36.514256 26104 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:36.514473 26104 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Initialized, Role: FOLLOWER
I20260806 22:43:36.514624 26104 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:36.514704 26104 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:36.514755 26104 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:36.514799 26104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:43:36.515792 26104 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:36.515888 26104 leader_election.cc:302] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b14bfd5bec4cde8beb4a0038c245b2; no voters:
I20260806 22:43:36.516052 26104 leader_election.cc:288] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:43:36.516103 26112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:43:36.516235 26112 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [term 3 LEADER]: Becoming Leader. State: Replica: b3b14bfd5bec4cde8beb4a0038c245b2, State: Running, Role: LEADER
I20260806 22:43:36.516316 26104 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:36.516381 26112 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } }
I20260806 22:43:36.516695 26117 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b14bfd5bec4cde8beb4a0038c245b2. Latest consensus state: current_term: 3 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:43:36.516708 26113 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b14bfd5bec4cde8beb4a0038c245b2" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37971 } } }
I20260806 22:43:36.516876 26117 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:36.516888 26113 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:36.517112 26134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:36.517997 26134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:36.518172 26134 catalog_manager.cc:1259] Loaded cluster ID: cc2d4eddb11148c8b02469c5184c4a22
I20260806 22:43:36.518261 26134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:36.519251 26134 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:36.519447 26134 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Loaded TSK: 0
I20260806 22:43:36.519484 26134 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3b14bfd5bec4cde8beb4a0038c245b2: Loaded TSK: 1
F20260806 22:43:36.519713 26134 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc2f70c59ad google::LogMessage::Fail() at ??:0
@ 0x7fc2f70c888a google::LogMessage::SendToLog() at ??:0
@ 0x7fc2f70c54de google::LogMessage::Flush() at ??:0
@ 0x7fc2f70c6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc2f9832104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc2f982fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc2ff01f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc2ff01f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc2ff01f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc2fefe5c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc2fefefad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc2f8f930d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc2f8f88954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc2f9dec6db start_thread at ??:0
@ 0x7fc2f5bc361f clone at ??:0
W20260806 22:43:36.594729 26108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:36.594923 26108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:36.594956 26108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:36.596148 26108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:36.596189 26108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:36.596223 26108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:36.597644 26108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:39721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=33655
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:36.597868 26108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:36.598078 26108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:36.603026 26184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.603039 26179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.603039 26180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:36.603399 26108 server_base.cc:1034] running on GCE node
I20260806 22:43:36.603600 26108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:36.603837 26108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:36.604988 26108 hybrid_clock.cc:648] HybridClock initialized: now 1786056216604970 us; error 35 us; skew 500 ppm
Aug 06 22:43:36 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:36.607451 26108 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:43:36.607905 26108 webserver.cc:460] Webserver started at http://127.0.186.193:33655/ using document root <none> and password file <none>
I20260806 22:43:36.608124 26108 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:36.608173 26108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:36.609423 26108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:36.610126 26192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.610322 26108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:36.610431 26108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "bd3504c82865411c94bb9d2bc2df087a"
format_stamp: "Formatted at 2026-08-06 22:42:54 on dist-test-slave-gjrz"
server_key: "1d16a6e86e4691096f6b4afb2bb2e830"
server_key_iv: "987530a13c243283c753563d7edec84e"
server_key_version: "encryptionkey@0"
I20260806 22:43:36.610883 26108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:36.627846 26108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:36.630162 26108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:36.630307 26108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:36.630606 26108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:36.631117 26204 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:43:36.632565 26108 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:43:36.632627 26108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:43:36.632660 26108 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:43:36.633579 26108 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:43:36.633627 26108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:43:36.633679 26204 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Bootstrap starting.
I20260806 22:43:36.634898 26204 log.cc:824] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:36.637727 26204 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:43:36.638019 26204 tablet_bootstrap.cc:492] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Bootstrap complete.
I20260806 22:43:36.638161 26204 ts_tablet_manager.cc:1397] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 22:43:36.639863 26204 raft_consensus.cc:348] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:36.640225 26204 raft_consensus.cc:729] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3504c82865411c94bb9d2bc2df087a, State: Initialized, Role: FOLLOWER
I20260806 22:43:36.640470 26204 consensus_queue.cc:260] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:36.640746 26204 ts_tablet_manager.cc:1428] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:36.641126 26108 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:39721
I20260806 22:43:36.641157 26365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:39721 every 8 connection(s)
I20260806 22:43:36.641512 26108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:36.645756 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 26108
I20260806 22:43:36.645996 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:43381
--local_ip_for_outbound_sockets=127.0.186.194
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=33391
--webserver_interface=127.0.186.194
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:36.710412 26371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:36.642200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:59473)
0806 22:43:36.642372 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.642417 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:36.642581 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:36.710209 (+ 67628us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
W20260806 22:43:36.710686 26366 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:37971 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:37971: Client connection negotiation failed: client connection to 127.0.186.254:37971: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:43:36.711071 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.710947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:34983)
0806 22:43:36.711000 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.711044 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:43:36.711390 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.711318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:57503)
0806 22:43:36.711346 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.711370 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 22:43:36.711513 26366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:43:36.727736 26374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:36.727918 26374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:36.727941 26374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:36.729077 26374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:36.729115 26374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:36.729140 26374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:36.730594 26374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:43381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=33391
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:36.730825 26374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:36.731025 26374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:36.735777 26381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.735885 26382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:36.735842 26374 server_base.cc:1034] running on GCE node
W20260806 22:43:36.735772 26384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:36.736168 26374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:36.736361 26374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:36.737499 26374 hybrid_clock.cc:648] HybridClock initialized: now 1786056216737496 us; error 30 us; skew 500 ppm
Aug 06 22:43:36 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:36.739856 26374 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:43:36.740231 26374 webserver.cc:460] Webserver started at http://127.0.186.194:33391/ using document root <none> and password file <none>
I20260806 22:43:36.740413 26374 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:36.740453 26374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:36.741640 26374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:36.742154 26396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.742288 26374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.742362 26374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "ae98ce2b311740efabc24f724791d417"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "572ac4c8c9b700ea34635f575540eb08"
server_key_iv: "43eb2d41f950940bf9cc16f9901ef42e"
server_key_version: "encryptionkey@0"
I20260806 22:43:36.742676 26374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:36.758369 26374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:36.760026 26374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:36.760145 26374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:36.760342 26374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:36.760615 26374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:36.760648 26374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.760666 26374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:36.760689 26374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.766137 26374 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43381
I20260806 22:43:36.766201 26561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43381 every 8 connection(s)
I20260806 22:43:36.766494 26374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:36.767436 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.767067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:44013)
0806 22:43:36.767203 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.767296 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
W20260806 22:43:36.767795 26562 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:37971 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:37971: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
I20260806 22:43:36.768016 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.767941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:53795)
0806 22:43:36.767976 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.768000 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 22:43:36.768220 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.768162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:57125)
0806 22:43:36.768190 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.768208 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 22:43:36.768301 26562 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:43:36.771602 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 26374
I20260806 22:43:36.771834 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:45873
--local_ip_for_outbound_sockets=127.0.186.195
--tserver_master_addrs=127.0.186.254:37971
--webserver_port=40489
--webserver_interface=127.0.186.195
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:36.852506 26566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:36.852697 26566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:36.852720 26566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:36.853823 26566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:36.853863 26566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:36.853888 26566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:36.855304 26566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:45873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=40489
--tserver_master_addrs=127.0.186.254:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:36.855509 26566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:36.855711 26566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:36.860287 26578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.860312 26575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:36.860297 26574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:36.860641 26566 server_base.cc:1034] running on GCE node
I20260806 22:43:36.860827 26566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:36.861043 26566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:36.862190 26566 hybrid_clock.cc:648] HybridClock initialized: now 1786056216862177 us; error 23 us; skew 500 ppm
Aug 06 22:43:36 dist-test-slave-gjrz krb5kdc[24324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:36.864502 26566 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:43:36.864917 26566 webserver.cc:460] Webserver started at http://127.0.186.195:40489/ using document root <none> and password file <none>
I20260806 22:43:36.865124 26566 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:36.865172 26566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:36.866370 26566 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:36.866874 26585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.867017 26566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.867112 26566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "4984ca7c1fb64c1bb18197932ddc212e"
format_stamp: "Formatted at 2026-08-06 22:42:55 on dist-test-slave-gjrz"
server_key: "614c78c8f06cc8d56dcdc0b88e73a885"
server_key_iv: "26f6b1374aaee3c8639c7b99a0489b48"
server_key_version: "encryptionkey@0"
I20260806 22:43:36.867450 26566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:36.878439 26566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:36.880162 26566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:36.880288 26566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:36.880486 26566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:36.880764 26566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:36.880796 26566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.880822 26566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:36.880839 26566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:36.886224 26566 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45873
I20260806 22:43:36.886286 26759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45873 every 8 connection(s)
I20260806 22:43:36.886576 26566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:36.887148 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 26566
I20260806 22:43:36.887637 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.887189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:58035)
0806 22:43:36.887359 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.887445 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
W20260806 22:43:36.888041 26763 heartbeater.cc:644] Failed to heartbeat to 127.0.186.254:37971 (0 consecutive failures): Network error: Failed to ping master at 127.0.186.254:37971: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
I20260806 22:43:36.888298 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.888213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:52351)
0806 22:43:36.888253 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.888280 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:43:36.888516 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:36.888461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:44841)
0806 22:43:36.888485 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:36.888503 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 22:43:36.888623 26763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:43:37.712008 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:37.711799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:43715)
0806 22:43:37.711910 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:37.711946 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260806 22:43:37.768833 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:37.768618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:39901)
0806 22:43:37.768752 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:37.768786 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260806 22:43:37.889132 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:37.888930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:46257)
0806 22:43:37.889049 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:37.889086 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260806 22:43:38.283600 26776 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:38.283720 26776 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:38.284051 26776 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
Aug 06 22:43:38 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.194@KRBTEST.COM
Aug 06 22:43:38 dist-test-slave-gjrz krb5kdc[24324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.195@KRBTEST.COM
I20260806 22:43:38.297850 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.284328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:36577 (local address 127.0.186.194:43381)
0806 22:43:38.284504 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:38.284517 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:38.284543 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:43:38.284979 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:38.284981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:38.285021 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:38.285149 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:38.285973 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.286898 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.287975 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.288360 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.288457 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:38.294243 (+ 5786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:38.294248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:38.294248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:38.294261 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:38.296144 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296511 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:38.296515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:38.296515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:38.296623 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296934 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:38.296935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:38.296935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:38.297218 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:38.297280 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:38.297620 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:43:38.297744 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":58,"threads_started":1}
I20260806 22:43:38.297850 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.284201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43381 (local address 127.0.186.193:36577)
0806 22:43:38.284321 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:38.284332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:38.284397 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:38.285223 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:38.285226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:38.285598 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:38.285835 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:38.285842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.287010 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.287012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:38.287887 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:38.287890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.288444 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.288445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:38.288561 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:38.292476 (+ 3915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:38.292486 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:38.294126 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:38.296219 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:38.296229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:38.296444 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:38.296730 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:38.296732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:38.296872 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:38.297302 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:38.297309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:38.297448 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:43:38.297552 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:43:38.297737 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":65,"thread_start_us":55,"threads_started":1}
I20260806 22:43:38.297894 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.284416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:59341 (local address 127.0.186.195:45873)
0806 22:43:38.284536 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:38.284553 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:38.284581 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:43:38.285031 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:38.285033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:38.285071 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:38.285197 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:38.286131 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.286995 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.287977 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.288358 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.288467 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:38.294690 (+ 6223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:38.294693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:38.294693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:38.294703 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:38.296367 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296739 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:38.296742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:38.296742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:38.296856 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:38.297111 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:38.297112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:38.297112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:38.297417 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:38.297508 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:38.297676 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:43:38.297797 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
I20260806 22:43:38.297945 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.284219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45873 (local address 127.0.186.193:59341)
0806 22:43:38.284325 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:38.284334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:38.284394 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:38.285225 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:38.285228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:38.285605 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:38.286007 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:38.286010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.287096 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.287097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:38.287889 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:38.287892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:38.288441 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:38.288442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:38.288559 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:38.292476 (+ 3917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:38.292486 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:38.294596 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:38.296439 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:38.296440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:38.296641 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:38.296959 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:38.296960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:38.296960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:38.297022 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:38.297494 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:38.297495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:38.297531 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:43:38.297600 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:43:38.297812 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":147,"thread_start_us":47,"threads_started":1}
W20260806 22:43:38.298771 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:38.298879 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:38.298944 26194 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:38.299065 26776 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:38.712587 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.712466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:49489)
0806 22:43:38.712511 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:38.712547 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:43:38.769454 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.769237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:40301)
0806 22:43:38.769350 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:38.769396 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260806 22:43:38.889771 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:38.889546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:57539)
0806 22:43:38.889696 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:38.889732 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260806 22:43:39.713331 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:39.713085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:38649)
0806 22:43:39.713239 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:39.713277 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260806 22:43:39.770182 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:39.769945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:49151)
0806 22:43:39.770104 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:39.770141 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260806 22:43:39.890416 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:39.890188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:55123)
0806 22:43:39.890341 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:39.890377 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260806 22:43:40.112635 26791 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:40.112738 26791 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:40.112901 26791 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
W20260806 22:43:40.113368 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:40.113468 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:40.113540 26197 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:40.113622 26791 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:40.713966 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:40.713766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:39487)
0806 22:43:40.713881 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:40.713911 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 22:43:40.770825 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:40.770623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:59105)
0806 22:43:40.770728 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:40.770789 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260806 22:43:40.891085 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:40.890861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:45115)
0806 22:43:40.891004 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:40.891045 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260806 22:43:41.714608 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:41.714383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:38539)
0806 22:43:41.714520 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:41.714559 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260806 22:43:41.771613 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:41.771373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:42575)
0806 22:43:41.771536 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:41.771573 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260806 22:43:41.819859 26800 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:41.819957 26800 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:41.820102 26800 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
W20260806 22:43:41.820468 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:41.820555 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:41.820613 26197 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:41.820693 26800 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:41.891685 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:41.891499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:59133)
0806 22:43:41.891609 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:41.891643 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260806 22:43:42.715183 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:42.714987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:54059)
0806 22:43:42.715101 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:42.715136 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260806 22:43:42.772190 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:42.772007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:57949)
0806 22:43:42.772119 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:42.772151 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260806 22:43:42.892313 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:42.892123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:51427)
0806 22:43:42.892238 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:42.892270 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260806 22:43:43.326462 26813 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:43.326548 26813 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:43.326716 26813 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
W20260806 22:43:43.327034 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:43.327142 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:43.327184 26194 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:43.327247 26813 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:43.715816 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:43.715588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:53587)
0806 22:43:43.715733 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:43.715769 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260806 22:43:43.772835 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:43.772625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:53397)
0806 22:43:43.772756 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:43.772792 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260806 22:43:43.892928 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:43.892709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:43447)
0806 22:43:43.892848 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:43.892884 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260806 22:43:44.716507 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:44.716278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:55195)
0806 22:43:44.716413 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:44.716452 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260806 22:43:44.773453 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:44.773241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:43767)
0806 22:43:44.773376 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:44.773414 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260806 22:43:44.893576 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:44.893353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:36661)
0806 22:43:44.893490 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:44.893528 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260806 22:43:45.717167 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:45.716944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:57861)
0806 22:43:45.717081 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:45.717119 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260806 22:43:45.774152 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:45.773927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:48775)
0806 22:43:45.774072 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:45.774109 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260806 22:43:45.894204 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:45.893992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:48681)
0806 22:43:45.894127 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:45.894162 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260806 22:43:45.955420 26827 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:45.955518 26827 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:45.955681 26827 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
W20260806 22:43:45.956027 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:45.956111 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:45.956171 26197 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:45.956239 26827 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:46.717778 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:46.717567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:47823)
0806 22:43:46.717694 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:46.717730 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260806 22:43:46.774855 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:46.774612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:51071)
0806 22:43:46.774730 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:46.774776 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260806 22:43:46.894817 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:46.894603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:59329)
0806 22:43:46.894735 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:46.894772 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 22:43:47.718381 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:47.718152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:54131)
0806 22:43:47.718294 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:47.718332 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260806 22:43:47.775487 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:47.775287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:56301)
0806 22:43:47.775404 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:47.775438 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260806 22:43:47.895524 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:47.895308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:38671)
0806 22:43:47.895442 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:47.895477 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 22:43:48.719028 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.718809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:51401)
0806 22:43:48.718946 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.718981 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260806 22:43:48.776106 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.775875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:39853)
0806 22:43:48.776034 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.776063 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260806 22:43:48.896183 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:48.895961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:37457)
0806 22:43:48.896101 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:48.896138 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260806 22:43:49.719728 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:49.719481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:56811)
0806 22:43:49.719641 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:49.719679 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260806 22:43:49.769456 26845 raft_consensus.cc:482] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:49.769553 26845 raft_consensus.cc:504] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4984ca7c1fb64c1bb18197932ddc212e" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45873 } } peers { permanent_uuid: "ae98ce2b311740efabc24f724791d417" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43381 } } peers { permanent_uuid: "bd3504c82865411c94bb9d2bc2df087a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 39721 } }
I20260806 22:43:49.769721 26845 leader_election.cc:288] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873), ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381)
W20260806 22:43:49.770117 26197 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4984ca7c1fb64c1bb18197932ddc212e (127.0.186.195:45873): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
W20260806 22:43:49.770205 26194 leader_election.cc:341] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae98ce2b311740efabc24f724791d417 (127.0.186.194:43381): Not found: Tablet not found: 088ef692af9246dcbaf51ff13624ccfa
I20260806 22:43:49.770239 26194 leader_election.cc:302] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd3504c82865411c94bb9d2bc2df087a; no voters: 4984ca7c1fb64c1bb18197932ddc212e, ae98ce2b311740efabc24f724791d417
I20260806 22:43:49.770298 26845 raft_consensus.cc:2726] T 088ef692af9246dcbaf51ff13624ccfa P bd3504c82865411c94bb9d2bc2df087a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:43:49.776736 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:49.776500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:42833)
0806 22:43:49.776646 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:49.776682 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260806 22:43:49.896919 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:49.896620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:33033)
0806 22:43:49.896783 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:49.896852 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
I20260806 22:43:50.720424 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:50.720153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:52727)
0806 22:43:50.720314 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:50.720373 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260806 22:43:50.777359 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:50.777131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:43009)
0806 22:43:50.777271 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:50.777308 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260806 22:43:50.897609 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:50.897387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:49019)
0806 22:43:50.897513 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:50.897563 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260806 22:43:51.721215 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:51.720990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.193:36651)
0806 22:43:51.721131 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:51.721169 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260806 22:43:51.778051 26855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:51.777810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.194:33739)
0806 22:43:51.777964 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:51.777999 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
I20260806 22:43:51.890882 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 26108
I20260806 22:43:51.905998 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:51.905787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:37971 (local address 127.0.186.195:35581)
0806 22:43:51.905905 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:51.905944 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.254:37971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260806 22:43:51.909785 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 26374
I20260806 22:43:51.922786 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 26566
2026-08-06T22:43:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58543 ms)
[----------] 30 tests from SecurityITest (234695 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:51 dist-test-slave-gjrz krb5kdc[26859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:51 dist-test-slave-gjrz krb5kdc[26859](info): set up 2 sockets
Aug 06 22:43:51 dist-test-slave-gjrz krb5kdc[26859](info): commencing operation
krb5kdc: starting...
W20260806 22:43:53.060169 747 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:43:53.174053 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46819
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:53.253554 26878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:53.253752 26878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:53.253782 26878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:53.254900 26878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:53.254940 26878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:53.254961 26878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:53.254983 26878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:53.255000 26878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:53.256497 26878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:53.256736 26878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:53.256935 26878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:43:53.261766 26878 server_base.cc:1034] running on GCE node
W20260806 22:43:53.261834 26885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.261852 26886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.261953 26894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:53.262300 26878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:53.262593 26878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:53.263722 26878 hybrid_clock.cc:648] HybridClock initialized: now 1786056233263709 us; error 30 us; skew 500 ppm
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:53.266063 26878 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:43:53.266501 26878 webserver.cc:460] Webserver started at http://127.0.186.254:35559/ using document root <none> and password file <none>
I20260806 22:43:53.266713 26878 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:53.266762 26878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:53.266891 26878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:53.267958 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "2ca45feed76c426cb8d30af3470632c4"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "6c813bdae324ed1cae8b859d207d6d19"
server_key_iv: "46b418d8022311338c69ea3b6582703a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.268306 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "2ca45feed76c426cb8d30af3470632c4"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "6c813bdae324ed1cae8b859d207d6d19"
server_key_iv: "46b418d8022311338c69ea3b6582703a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.269521 26878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:43:53.270159 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.270313 26878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:53.270387 26878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "2ca45feed76c426cb8d30af3470632c4"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "6c813bdae324ed1cae8b859d207d6d19"
server_key_iv: "46b418d8022311338c69ea3b6582703a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.270447 26878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:53.285830 26878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:53.287619 26878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:53.287747 26878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:53.291555 26878 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:46819
I20260806 22:43:53.291589 26992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:46819 every 8 connection(s)
I20260806 22:43:53.291894 26878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:53.292939 26993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:53.295183 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: Bootstrap starting.
I20260806 22:43:53.295782 26993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:53.296021 26993 log.cc:824] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:53.296627 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: No bootstrap required, opened a new log
I20260806 22:43:53.298218 26993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } }
I20260806 22:43:53.298414 26993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:53.298480 26993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca45feed76c426cb8d30af3470632c4, State: Initialized, Role: FOLLOWER
I20260806 22:43:53.298609 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 26878
I20260806 22:43:53.298640 26993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } }
I20260806 22:43:53.298712 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:53.298720 26993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:53.298748 26993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:53.298787 26993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:53.298913 747 external_mini_cluster.cc:1428] Setting key 46ab11f0c90ec73684a1afb70a574733
I20260806 22:43:53.299511 26993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } }
I20260806 22:43:53.299633 26993 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ca45feed76c426cb8d30af3470632c4; no voters:
I20260806 22:43:53.299784 26993 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:53.299836 27000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:53.300071 27000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ca45feed76c426cb8d30af3470632c4, State: Running, Role: LEADER
I20260806 22:43:53.300258 27000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } }
I20260806 22:43:53.300720 27005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca45feed76c426cb8d30af3470632c4. Latest consensus state: current_term: 1 leader_uuid: "2ca45feed76c426cb8d30af3470632c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } } }
I20260806 22:43:53.300854 27004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ca45feed76c426cb8d30af3470632c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca45feed76c426cb8d30af3470632c4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46819 } } }
I20260806 22:43:53.300961 27004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:53.300097 26993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:53.301329 27010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:53.301816 27010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:53.300870 27005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:53.310880 27010 catalog_manager.cc:1347] Generated new cluster ID: e97875e0433c4335a8ed8dc72a51e2c3
I20260806 22:43:53.310941 27010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:53.315127 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.299630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.0.186.254:46819)
0806 22:43:53.299831 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.299840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.299873 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:43:53.299993 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.300001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.300054 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.300213 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:53.300959 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.302356 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.303578 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.304204 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.304314 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.307244 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:53.307248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:53.307249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:53.307286 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:53.311011 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.311456 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.311460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.311460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.311589 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.311923 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.311924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.311924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.312238 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:53.314691 (+ 2453us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.314781 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.314919 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260806 22:43:53.332891 27010 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:53.333626 27010 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:53.336361 27010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: Generated new TSK 0
I20260806 22:43:53.336588 27010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:43:53.371369 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:53.476689 27036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:53.476876 27036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:53.476898 27036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:53.477989 27036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:53.478024 27036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:53.478053 27036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:53.479540 27036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:53.479722 27036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:53.479893 27036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:53.484601 27048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:53.484644 27036 server_base.cc:1034] running on GCE node
W20260806 22:43:53.484545 27049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.484553 27053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:53.484989 27036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:53.485211 27036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:53.486354 27036 hybrid_clock.cc:648] HybridClock initialized: now 1786056233486340 us; error 34 us; skew 500 ppm
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:53.488801 27036 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:43:53.489187 27036 webserver.cc:460] Webserver started at http://127.0.186.193:36855/ using document root <none> and password file <none>
I20260806 22:43:53.489360 27036 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:53.489396 27036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:53.489483 27036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:53.490401 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "1cf74f6521614df5afff2c2e74b354e7"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "1a5a3b53042867b6478e9a5d5d013e93"
server_key_iv: "b2050d7f0cddf345207344cf590d7b5a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.490715 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "1cf74f6521614df5afff2c2e74b354e7"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "1a5a3b53042867b6478e9a5d5d013e93"
server_key_iv: "b2050d7f0cddf345207344cf590d7b5a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.492353 27036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:53.493201 27063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.493373 27036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:53.493462 27036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "1cf74f6521614df5afff2c2e74b354e7"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "1a5a3b53042867b6478e9a5d5d013e93"
server_key_iv: "b2050d7f0cddf345207344cf590d7b5a"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.493532 27036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:53.513320 27036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:53.515106 27036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:53.515246 27036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:53.515497 27036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:53.515799 27036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:53.515832 27036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.515861 27036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:53.515883 27036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.521384 27036 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:44235
I20260806 22:43:53.521435 27233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:44235 every 8 connection(s)
I20260806 22:43:53.521737 27036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:53.527235 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 27036
I20260806 22:43:53.527374 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:53.527580 747 external_mini_cluster.cc:1428] Setting key 307011792e024d9c6da4b077772b14b9
I20260806 22:43:53.530725 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.522439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:44697 (local address 127.0.186.254:46819)
0806 22:43:53.522504 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.522506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.522662 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:43:53.523601 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.523602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.523616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.523672 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:53.524286 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.524807 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.525728 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.525923 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.525971 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.528080 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:53.528081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:53.528081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:53.528089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:53.529385 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.529845 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.529846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.529846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.529917 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.530136 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.530137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.530137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.530268 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:53.530337 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.530542 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.530638 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:43:53.530776 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.522392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46819 (local address 127.0.186.193:44697)
0806 22:43:53.522556 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.522594 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.523490 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.523775 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.523779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.524029 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:53.524209 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.524213 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.524872 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.524873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.525664 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.525667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.525984 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.525985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.526081 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.526530 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:53.526538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:53.527995 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:53.529471 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.529479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.529481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.529762 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.529981 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.529981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.529982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.530081 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.530325 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:53.530328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:53.530431 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.530509 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.530625 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260806 22:43:53.531239 27234 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46819
I20260806 22:43:53.531354 27234 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:53.531525 27234 heartbeater.cc:511] Master 127.0.186.254:46819 requested a full tablet report, sending...
I20260806 22:43:53.532017 26940 ts_manager.cc:194] Registered new tserver with Master: 1cf74f6521614df5afff2c2e74b354e7 (127.0.186.193:44235)
I20260806 22:43:53.532603 26940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:44697
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:43:53.551815 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:53.631978 27239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:53.632169 27239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:53.632200 27239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:53.633317 27239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:53.633358 27239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:53.633394 27239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:53.634857 27239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:53.635095 27239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:53.635316 27239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:53.640091 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.640103 27253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.640091 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:53.640359 27239 server_base.cc:1034] running on GCE node
I20260806 22:43:53.640565 27239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:53.640789 27239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:53.641933 27239 hybrid_clock.cc:648] HybridClock initialized: now 1786056233641927 us; error 34 us; skew 500 ppm
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:53.644357 27239 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:43:53.644745 27239 webserver.cc:460] Webserver started at http://127.0.186.194:40091/ using document root <none> and password file <none>
I20260806 22:43:53.644948 27239 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:53.644990 27239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:53.645118 27239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:53.646113 27239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "dc53bc520f43407f9805f81a41168958"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "c8552b09077641537e72ca25db32ca35"
server_key_iv: "a0509e8740aba601dd44cb9fbb117695"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.646435 27239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "dc53bc520f43407f9805f81a41168958"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "c8552b09077641537e72ca25db32ca35"
server_key_iv: "a0509e8740aba601dd44cb9fbb117695"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.647799 27239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:53.648587 27266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.648732 27239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:43:53.648821 27239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "dc53bc520f43407f9805f81a41168958"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "c8552b09077641537e72ca25db32ca35"
server_key_iv: "a0509e8740aba601dd44cb9fbb117695"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.648926 27239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:53.662787 27239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:53.664578 27239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:53.664695 27239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:53.664898 27239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:53.665194 27239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:53.665227 27239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.665247 27239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:53.665270 27239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.670766 27239 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:38501
I20260806 22:43:53.670821 27436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:38501 every 8 connection(s)
I20260806 22:43:53.671098 27239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:53.676385 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 27239
I20260806 22:43:53.676515 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:53.676708 747 external_mini_cluster.cc:1428] Setting key e27f01232d5c6b795458e00ff118e01f
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:53.680119 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.671780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:36629 (local address 127.0.186.254:46819)
0806 22:43:53.671834 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.671836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.672020 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:43:53.672925 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.672926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.672937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.672993 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:53.673537 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.674082 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.675023 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.675232 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.675311 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.677458 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:53.677460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:53.677460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:53.677468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:53.678836 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.679194 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.679195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.679195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.679282 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.679551 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.679551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.679552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.679652 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:53.679702 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.679976 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.680057 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:43:53.680337 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.671758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46819 (local address 127.0.186.194:36629)
0806 22:43:53.671909 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.671943 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.672813 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.673053 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.673056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.673292 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:53.673476 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.673480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.674156 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.674156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.674929 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.674931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.675315 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.675316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.675412 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.675792 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:53.675798 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:53.677379 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:53.678910 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.678917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.678918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.679131 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.679346 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.679347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.679348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.679487 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.679705 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:53.679709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:53.679860 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.679971 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.680126 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260806 22:43:53.680851 27437 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46819
I20260806 22:43:53.680944 27437 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:53.681203 27437 heartbeater.cc:511] Master 127.0.186.254:46819 requested a full tablet report, sending...
I20260806 22:43:53.681571 26940 ts_manager.cc:194] Registered new tserver with Master: dc53bc520f43407f9805f81a41168958 (127.0.186.194:38501)
I20260806 22:43:53.681943 26940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:36629
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:43:53.700184 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:36929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:53.780928 27447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:53.781131 27447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:53.781154 27447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:53.782259 27447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:53.782294 27447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:53.782318 27447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:53.783787 27447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:36929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:46819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:53.783981 27447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:53.784160 27447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:53.788838 27456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.788848 27457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:53.788914 27459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:53.788966 27447 server_base.cc:1034] running on GCE node
I20260806 22:43:53.789264 27447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:53.789461 27447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:53.790635 27447 hybrid_clock.cc:648] HybridClock initialized: now 1786056233790622 us; error 28 us; skew 500 ppm
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:53.793138 27447 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:43:53.793541 27447 webserver.cc:460] Webserver started at http://127.0.186.195:46191/ using document root <none> and password file <none>
I20260806 22:43:53.793751 27447 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:53.793793 27447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:53.793926 27447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:53.794900 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "e05804bf6bb74a58b074d9cd72007c4d"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "ab5c36bb7180dcb56420fd5229ea63ec"
server_key_iv: "2db309d8a4d66aff45a56e18d6f48842"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.795233 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "e05804bf6bb74a58b074d9cd72007c4d"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "ab5c36bb7180dcb56420fd5229ea63ec"
server_key_iv: "2db309d8a4d66aff45a56e18d6f48842"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.796556 27447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:53.797243 27477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.797381 27447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:53.797458 27447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "e05804bf6bb74a58b074d9cd72007c4d"
format_stamp: "Formatted at 2026-08-06 22:43:53 on dist-test-slave-gjrz"
server_key: "ab5c36bb7180dcb56420fd5229ea63ec"
server_key_iv: "2db309d8a4d66aff45a56e18d6f48842"
server_key_version: "encryptionkey@0"
I20260806 22:43:53.797529 27447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:53.838397 27447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:53.840493 27447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:53.840621 27447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:53.840857 27447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:53.841194 27447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:53.841230 27447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.841260 27447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:53.841284 27447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:53.846825 27447 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:33049
I20260806 22:43:53.846889 27648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:33049 every 8 connection(s)
I20260806 22:43:53.847183 27447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:43:53 dist-test-slave-gjrz krb5kdc[26859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:53.855451 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 27447
I20260806 22:43:53.855543 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:53.855738 747 external_mini_cluster.cc:1428] Setting key 81761c915baaf69f4e0ad77803c049c6
I20260806 22:43:53.855850 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.847903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:42413 (local address 127.0.186.254:46819)
0806 22:43:53.847945 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.847947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.848151 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:43:53.849120 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.849121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.849133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.849185 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:53.849743 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.850254 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.851195 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.851435 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.851483 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.853520 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:53.853522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:53.853522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:53.853529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:53.854606 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.854953 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.854954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.854954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.854999 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.855176 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.855176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.855177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.855307 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:53.855366 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.855700 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.855764 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:43:53.856070 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.847869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46819 (local address 127.0.186.195:42413)
0806 22:43:53.848015 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.848069 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.849016 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.849244 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.849246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.849489 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:53.849680 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.849685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.850319 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.850320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.851121 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.851123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.851501 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.851502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.851625 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.852076 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:53.852084 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:53.853419 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:53.854685 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.854693 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.854694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.854901 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.855048 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:53.855048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:53.855049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:53.855133 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:53.855374 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:53.855382 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:53.855557 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.855678 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.855843 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260806 22:43:53.856483 27652 heartbeater.cc:347] Connected to a master server at 127.0.186.254:46819
I20260806 22:43:53.856587 27652 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:53.856772 27652 heartbeater.cc:511] Master 127.0.186.254:46819 requested a full tablet report, sending...
I20260806 22:43:53.857098 26939 ts_manager.cc:194] Registered new tserver with Master: e05804bf6bb74a58b074d9cd72007c4d (127.0.186.195:33049)
I20260806 22:43:53.857445 26939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:42413
I20260806 22:43:53.867449 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:53.873759 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.868649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.0.186.254:46819)
0806 22:43:53.868712 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.868714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.868797 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:43:53.868842 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.868842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.868851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.868890 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:53.869364 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.869815 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.870455 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.870637 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.870680 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.871688 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:53.871689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:53.871690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:53.871695 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:53.872890 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.873167 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.873167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.873168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.873216 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:53.873337 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:53.873337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:53.873338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:53.873461 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:53.873546 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.873629 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.873688 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:43:53.875101 26939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:53.876688 26939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:53.883189 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.878878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:33049 (local address 127.0.0.1:40824)
0806 22:43:53.879388 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.879399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.879479 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.879978 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.879979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.879981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:53.880021 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.880022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.881137 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.881137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.882025 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.882028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.882902 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.882903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.882971 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.882983 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.883061 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.883120 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":60,"threads_started":1}
I20260806 22:43:53.883205 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.879021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.0.186.195:33049)
0806 22:43:53.879180 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.879184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.879836 (+ 652us) server_negotiation.cc:407] Connection header received
0806 22:43:53.879878 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.879880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.879895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.879972 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:53.880069 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.881051 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.882104 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.882806 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.882886 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.882939 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.883079 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.883123 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260806 22:43:53.884138 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.878709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44235 (local address 127.0.0.1:41738)
0806 22:43:53.878998 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.879039 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.879101 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.879410 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.879413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.879416 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:53.879510 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.879512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.880788 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.880788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.882022 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.882025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.883907 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.883908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.883967 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.883971 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.884031 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.884070 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":4,"thread_start_us":72,"threads_started":1}
I20260806 22:43:53.884157 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.878843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.0.186.193:44235)
0806 22:43:53.879211 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.879214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.879224 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:43:53.879291 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.879294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.879309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.879391 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:53.879616 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.880695 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.882976 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.883761 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.883817 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.883875 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.884047 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.884087 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":100,"threads_started":1}
I20260806 22:43:53.884089 27540 tablet_service.cc:1467] Processing CreateTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 (DEFAULT_TABLE table=test-table [id=b2f130551a7f405b98e8507b3342022f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:53.884451 27540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f576cdd32d22481ca21d6c7f0f11ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:53.884873 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.878694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:38501 (local address 127.0.0.1:58862)
0806 22:43:53.878998 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:53.879039 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:53.879101 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:53.880823 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:53.880824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:53.880827 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:53.880877 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.880879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.882235 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.882236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.883588 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:53.883592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.884645 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.884647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:53.884709 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.884712 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:53.884767 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:43:53.884804 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":50,"threads_started":1}
I20260806 22:43:53.884775 27130 tablet_service.cc:1467] Processing CreateTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 (DEFAULT_TABLE table=test-table [id=b2f130551a7f405b98e8507b3342022f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:53.885089 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:53.878810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.0.186.194:38501)
0806 22:43:53.879605 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:53.879609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:53.879618 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:53.879655 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:53.879657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:53.879672 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:53.879750 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:53.880956 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.882155 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.883669 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:53.884468 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:53.884551 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:53.884607 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:53.884801 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:43:53.884847 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":84,"threads_started":1}
I20260806 22:43:53.885113 27130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f576cdd32d22481ca21d6c7f0f11ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:53.887159 27675 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Bootstrap starting.
I20260806 22:43:53.887622 27676 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Bootstrap starting.
I20260806 22:43:53.887988 27675 tablet_bootstrap.cc:654] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:53.888284 27676 tablet_bootstrap.cc:654] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:53.888242 27337 tablet_service.cc:1467] Processing CreateTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 (DEFAULT_TABLE table=test-table [id=b2f130551a7f405b98e8507b3342022f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:53.888521 27337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f576cdd32d22481ca21d6c7f0f11ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:53.888612 27675 log.cc:824] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:53.888844 27676 log.cc:824] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:53.889397 27675 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: No bootstrap required, opened a new log
I20260806 22:43:53.889456 27675 ts_tablet_manager.cc:1397] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:53.889704 27676 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: No bootstrap required, opened a new log
I20260806 22:43:53.889810 27676 ts_tablet_manager.cc:1397] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:53.890810 27675 raft_consensus.cc:348] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.890949 27675 raft_consensus.cc:374] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:53.890990 27675 raft_consensus.cc:729] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05804bf6bb74a58b074d9cd72007c4d, State: Initialized, Role: FOLLOWER
I20260806 22:43:53.891114 27675 consensus_queue.cc:260] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.891419 27675 ts_tablet_manager.cc:1428] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:53.891562 27652 heartbeater.cc:503] Master 127.0.186.254:46819 was elected leader, sending a full tablet report...
I20260806 22:43:53.891707 27683 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Bootstrap starting.
I20260806 22:43:53.892352 27683 tablet_bootstrap.cc:654] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:53.892292 27676 raft_consensus.cc:348] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.892460 27676 raft_consensus.cc:374] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:53.892513 27676 raft_consensus.cc:729] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf74f6521614df5afff2c2e74b354e7, State: Initialized, Role: FOLLOWER
I20260806 22:43:53.892625 27676 consensus_queue.cc:260] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.892891 27676 ts_tablet_manager.cc:1428] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:43:53.892917 27234 heartbeater.cc:503] Master 127.0.186.254:46819 was elected leader, sending a full tablet report...
I20260806 22:43:53.892933 27683 log.cc:824] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:53.893712 27683 tablet_bootstrap.cc:492] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: No bootstrap required, opened a new log
I20260806 22:43:53.893786 27683 ts_tablet_manager.cc:1397] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:53.895066 27683 raft_consensus.cc:348] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.895196 27683 raft_consensus.cc:374] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:53.895216 27683 raft_consensus.cc:729] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc53bc520f43407f9805f81a41168958, State: Initialized, Role: FOLLOWER
I20260806 22:43:53.895331 27683 consensus_queue.cc:260] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:53.895613 27683 ts_tablet_manager.cc:1428] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:53.895632 27437 heartbeater.cc:503] Master 127.0.186.254:46819 was elected leader, sending a full tablet report...
W20260806 22:43:53.921686 27438 tablet.cc:2369] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:54.022485 27235 tablet.cc:2369] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:54.097791 27656 tablet.cc:2369] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:54.149523 27682 raft_consensus.cc:482] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:54.149646 27682 raft_consensus.cc:504] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:54.150055 27682 leader_election.cc:288] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc53bc520f43407f9805f81a41168958 (127.0.186.194:38501), 1cf74f6521614df5afff2c2e74b354e7 (127.0.186.193:44235)
I20260806 22:43:54.153931 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.150313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:38189 (local address 127.0.186.194:38501)
0806 22:43:54.150358 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.150360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.150536 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:43:54.150575 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.150576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.150586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.150634 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.150752 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.151523 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.152991 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.153557 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.153630 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.153638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.153818 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.153867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:43:54.153931 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.150269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44235 (local address 127.0.186.195:35745)
0806 22:43:54.150322 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.150335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.150385 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.150578 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.150579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.150581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.150657 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.150659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.151599 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.151600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.152887 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.152891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.153632 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.153633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.153722 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.153724 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.153778 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.153821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":46}
I20260806 22:43:54.153937 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.150269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:38501 (local address 127.0.186.195:38189)
0806 22:43:54.150463 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.150473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.150530 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.150643 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.150644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.150645 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.150695 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.150696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.151599 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.151600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.152887 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.152891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.153632 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.153633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.153719 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.153724 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.153778 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.153821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":63,"thread_start_us":117,"threads_started":1}
I20260806 22:43:54.153931 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.150344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:35745 (local address 127.0.186.193:44235)
0806 22:43:54.150387 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.150389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.150399 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:43:54.150484 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.150485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.150493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.150549 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.150727 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.151523 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.152991 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.153555 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.153630 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.153638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.153818 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.153861 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:43:54.154345 27165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f576cdd32d22481ca21d6c7f0f11ba96" candidate_uuid: "e05804bf6bb74a58b074d9cd72007c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf74f6521614df5afff2c2e74b354e7" is_pre_election: true
I20260806 22:43:54.154362 27367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f576cdd32d22481ca21d6c7f0f11ba96" candidate_uuid: "e05804bf6bb74a58b074d9cd72007c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc53bc520f43407f9805f81a41168958" is_pre_election: true
I20260806 22:43:54.154511 27367 raft_consensus.cc:2454] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e05804bf6bb74a58b074d9cd72007c4d in term 0.
I20260806 22:43:54.154511 27165 raft_consensus.cc:2454] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e05804bf6bb74a58b074d9cd72007c4d in term 0.
I20260806 22:43:54.154750 27479 leader_election.cc:302] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc53bc520f43407f9805f81a41168958, e05804bf6bb74a58b074d9cd72007c4d; no voters:
I20260806 22:43:54.154886 27682 raft_consensus.cc:2781] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:54.154943 27682 raft_consensus.cc:482] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:54.154968 27682 raft_consensus.cc:3037] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.155726 27682 raft_consensus.cc:504] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:54.155838 27682 leader_election.cc:288] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [CANDIDATE]: Term 1 election: Requested vote from peers dc53bc520f43407f9805f81a41168958 (127.0.186.194:38501), 1cf74f6521614df5afff2c2e74b354e7 (127.0.186.193:44235)
I20260806 22:43:54.156003 27367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f576cdd32d22481ca21d6c7f0f11ba96" candidate_uuid: "e05804bf6bb74a58b074d9cd72007c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc53bc520f43407f9805f81a41168958"
I20260806 22:43:54.155992 27165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f576cdd32d22481ca21d6c7f0f11ba96" candidate_uuid: "e05804bf6bb74a58b074d9cd72007c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf74f6521614df5afff2c2e74b354e7"
I20260806 22:43:54.156077 27165 raft_consensus.cc:3037] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.156077 27367 raft_consensus.cc:3037] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.156916 27165 raft_consensus.cc:2454] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e05804bf6bb74a58b074d9cd72007c4d in term 1.
I20260806 22:43:54.156916 27367 raft_consensus.cc:2454] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e05804bf6bb74a58b074d9cd72007c4d in term 1.
I20260806 22:43:54.157075 27479 leader_election.cc:302] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc53bc520f43407f9805f81a41168958, e05804bf6bb74a58b074d9cd72007c4d; no voters:
I20260806 22:43:54.157150 27682 raft_consensus.cc:2781] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:54.157277 27682 raft_consensus.cc:686] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 1 LEADER]: Becoming Leader. State: Replica: e05804bf6bb74a58b074d9cd72007c4d, State: Running, Role: LEADER
I20260806 22:43:54.157356 27682 consensus_queue.cc:237] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } }
I20260806 22:43:54.158154 26939 catalog_manager.cc:5495] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to e05804bf6bb74a58b074d9cd72007c4d (127.0.186.195). New cstate: current_term: 1 leader_uuid: "e05804bf6bb74a58b074d9cd72007c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e05804bf6bb74a58b074d9cd72007c4d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33049 } health_report { overall_health: HEALTHY } } }
I20260806 22:43:54.163805 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.161632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.0.186.195:33049)
0806 22:43:54.161674 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.161676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.161683 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:43:54.161739 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.161739 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.161755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.161810 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:54.161937 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.162362 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.163078 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.163253 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.163326 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.163484 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:54.163585 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:54.163653 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.163700 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.163721 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:43:54.165376 27367 raft_consensus.cc:1261] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 1 FOLLOWER]: Refusing update from remote peer e05804bf6bb74a58b074d9cd72007c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:54.165406 27165 raft_consensus.cc:1261] T f576cdd32d22481ca21d6c7f0f11ba96 P 1cf74f6521614df5afff2c2e74b354e7 [term 1 FOLLOWER]: Refusing update from remote peer e05804bf6bb74a58b074d9cd72007c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:54.165685 27682 consensus_queue.cc:1035] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc53bc520f43407f9805f81a41168958" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 38501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:54.165809 27689 consensus_queue.cc:1035] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cf74f6521614df5afff2c2e74b354e7" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:54.167981 27693 mvcc.cc:205] Tried to move back new op lower bound from 7315686335139209216 to 7315686335109066752. Current Snapshot: MvccSnapshot[applied={T|T < 7315686335139209216}]
I20260806 22:43:54.168118 27694 mvcc.cc:205] Tried to move back new op lower bound from 7315686335139209216 to 7315686335109066752. Current Snapshot: MvccSnapshot[applied={T|T < 7315686335139209216}]
I20260806 22:43:54.168761 27695 mvcc.cc:205] Tried to move back new op lower bound from 7315686335139209216 to 7315686335109066752. Current Snapshot: MvccSnapshot[applied={T|T < 7315686335139209216 or (T in {7315686335139209216})}]
I20260806 22:43:54.170615 26939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:54.170711 26939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:54.171411 26939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca45feed76c426cb8d30af3470632c4: Sending DeleteTablet for 3 replicas of tablet f576cdd32d22481ca21d6c7f0f11ba96
I20260806 22:43:54.171736 27337 tablet_service.cc:1514] Processing DeleteTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:58862
I20260806 22:43:54.171758 27540 tablet_service.cc:1514] Processing DeleteTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40824
I20260806 22:43:54.171993 27130 tablet_service.cc:1514] Processing DeleteTablet for tablet f576cdd32d22481ca21d6c7f0f11ba96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:41738
I20260806 22:43:54.172036 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 27036
I20260806 22:43:54.183476 27704 tablet_replica.cc:330] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: stopping tablet replica
I20260806 22:43:54.183601 27704 raft_consensus.cc:2229] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:54.183722 27704 raft_consensus.cc:2258] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:54.183909 27706 tablet_replica.cc:330] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: stopping tablet replica
I20260806 22:43:54.184058 27706 raft_consensus.cc:2229] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:43:54.184235 27706 raft_consensus.cc:2258] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:54.184257 27704 ts_tablet_manager.cc:1905] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:54.184682 27706 ts_tablet_manager.cc:1905] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:54.186091 27704 ts_tablet_manager.cc:1918] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:54.186173 27706 ts_tablet_manager.cc:1918] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:54.186239 27706 log.cc:1196] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/f576cdd32d22481ca21d6c7f0f11ba96
I20260806 22:43:54.186317 27704 log.cc:1196] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/f576cdd32d22481ca21d6c7f0f11ba96
I20260806 22:43:54.186537 27706 ts_tablet_manager.cc:1939] T f576cdd32d22481ca21d6c7f0f11ba96 P e05804bf6bb74a58b074d9cd72007c4d: Deleting consensus metadata
I20260806 22:43:54.186663 27704 ts_tablet_manager.cc:1939] T f576cdd32d22481ca21d6c7f0f11ba96 P dc53bc520f43407f9805f81a41168958: Deleting consensus metadata
I20260806 22:43:54.186947 26910 catalog_manager.cc:4841] TS e05804bf6bb74a58b074d9cd72007c4d (127.0.186.195:33049): tablet f576cdd32d22481ca21d6c7f0f11ba96 (table test-table [id=b2f130551a7f405b98e8507b3342022f]) successfully deleted
I20260806 22:43:54.187042 26908 catalog_manager.cc:4841] TS dc53bc520f43407f9805f81a41168958 (127.0.186.194:38501): tablet f576cdd32d22481ca21d6c7f0f11ba96 (table test-table [id=b2f130551a7f405b98e8507b3342022f]) successfully deleted
W20260806 22:43:54.188395 26910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:44235 (error 108)
I20260806 22:43:54.188889 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.188776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44235 (local address 127.0.0.1:41752)
0806 22:43:54.188823 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.188858 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:43:54.188982 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 27239
W20260806 22:43:54.188997 26910 catalog_manager.cc:4584] TS 1cf74f6521614df5afff2c2e74b354e7 (127.0.186.193:44235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f576cdd32d22481ca21d6c7f0f11ba96: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44235: connect: Connection refused (error 111)
I20260806 22:43:54.197685 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 27447
I20260806 22:43:54.207933 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 26878
2026-08-06T22:43:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2284 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:54Z Disabled control of system clock
I20260806 22:43:54.241050 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:34983
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:34983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:54.323130 27713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:54.323379 27713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:54.323405 27713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:54.324517 27713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:54.324551 27713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:54.324563 27713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:54.324574 27713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:54.324584 27713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:54.325870 27713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:34983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:34983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:54.326059 27713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:54.326283 27713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:54.330842 27728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.330845 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.330853 27720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:54.331060 27713 server_base.cc:1034] running on GCE node
I20260806 22:43:54.331310 27713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:54.331583 27713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:54.332741 27713 hybrid_clock.cc:648] HybridClock initialized: now 1786056234332725 us; error 28 us; skew 500 ppm
I20260806 22:43:54.333930 27713 webserver.cc:460] Webserver started at http://127.0.186.254:43977/ using document root <none> and password file <none>
I20260806 22:43:54.334136 27713 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:54.334177 27713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:54.334267 27713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:54.335310 27713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "bbf82a58d32e4579813fa98af7b36849"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "fec09f44eda041da5e28aea936b0df7a"
server_key_iv: "c63059971320c08187d38a295e8ba0e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.335628 27713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "bbf82a58d32e4579813fa98af7b36849"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "fec09f44eda041da5e28aea936b0df7a"
server_key_iv: "c63059971320c08187d38a295e8ba0e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.336778 27713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:54.337448 27737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.337617 27713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:54.337708 27713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "bbf82a58d32e4579813fa98af7b36849"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "fec09f44eda041da5e28aea936b0df7a"
server_key_iv: "c63059971320c08187d38a295e8ba0e2"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.337766 27713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:54.354662 27713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:54.354941 27713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:54.355072 27713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:54.358779 27713 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:34983
I20260806 22:43:54.358814 27824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:34983 every 8 connection(s)
I20260806 22:43:54.359123 27713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:54.360083 27826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:54.362337 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: Bootstrap starting.
I20260806 22:43:54.362910 27826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:54.363178 27826 log.cc:824] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:54.363829 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: No bootstrap required, opened a new log
I20260806 22:43:54.365418 27826 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } }
I20260806 22:43:54.365540 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 27713
I20260806 22:43:54.365571 27826 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:54.365593 27826 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbf82a58d32e4579813fa98af7b36849, State: Initialized, Role: FOLLOWER
I20260806 22:43:54.365638 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:54.365731 27826 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } }
I20260806 22:43:54.365796 27826 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:54.365816 27826 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:54.365849 747 external_mini_cluster.cc:1428] Setting key d4eab56ec78a6bf0740284831c9af550
I20260806 22:43:54.365860 27826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.366426 27826 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } }
I20260806 22:43:54.366547 27826 leader_election.cc:302] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbf82a58d32e4579813fa98af7b36849; no voters:
I20260806 22:43:54.366704 27826 leader_election.cc:288] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:54.366752 27833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:54.366899 27833 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [term 1 LEADER]: Becoming Leader. State: Replica: bbf82a58d32e4579813fa98af7b36849, State: Running, Role: LEADER
I20260806 22:43:54.366966 27826 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:54.367045 27833 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } }
I20260806 22:43:54.367431 27835 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbf82a58d32e4579813fa98af7b36849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } } }
I20260806 22:43:54.367529 27835 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:54.367744 27836 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbf82a58d32e4579813fa98af7b36849. Latest consensus state: current_term: 1 leader_uuid: "bbf82a58d32e4579813fa98af7b36849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf82a58d32e4579813fa98af7b36849" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 34983 } } }
I20260806 22:43:54.368098 27836 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:54.368263 27842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:54.368793 27842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:54.370325 27842 catalog_manager.cc:1347] Generated new cluster ID: 338c75be75b74192b6300b969a3449cf
I20260806 22:43:54.370369 27842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:54.371884 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.366327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.0.186.254:34983)
0806 22:43:54.366555 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.366564 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.366596 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:43:54.366711 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.366716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.366777 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.367157 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:54.367291 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.368879 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.369768 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.370360 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.370520 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.371176 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:54.371180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:54.371180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:54.371405 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:54.371448 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:54.371465 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:54.371535 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.371591 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.371673 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260806 22:43:54.376936 27842 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:54.377427 27842 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:54.382099 27842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: Generated new TSK 0
I20260806 22:43:54.382259 27842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:43:54.384464 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--builtin_ntp_servers=127.0.186.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:54.465623 27862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:54.465819 27862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:54.465848 27862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:54.466962 27862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:54.467001 27862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:54.467033 27862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:54.468494 27862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:54.468712 27862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:54.468981 27862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:54.474602 27869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.474632 27872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.474644 27870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:54.475205 27862 server_base.cc:1034] running on GCE node
I20260806 22:43:54.475426 27862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:54.475651 27862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:54.476811 27862 hybrid_clock.cc:648] HybridClock initialized: now 1786056234476800 us; error 33 us; skew 500 ppm
I20260806 22:43:54.478014 27862 webserver.cc:460] Webserver started at http://127.0.186.193:36029/ using document root <none> and password file <none>
I20260806 22:43:54.478219 27862 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:54.478266 27862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:54.478387 27862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:54.479442 27862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "07aeb803f4624ab1b5acb736ae48a315"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "d02431462a2cdc28200788ed8a7eef9d"
server_key_iv: "e8f5b3f33ec9038b44b383cd5e94e9cc"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.479781 27862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "07aeb803f4624ab1b5acb736ae48a315"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "d02431462a2cdc28200788ed8a7eef9d"
server_key_iv: "e8f5b3f33ec9038b44b383cd5e94e9cc"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.480963 27862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:54.481640 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.481820 27862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:54.481933 27862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "07aeb803f4624ab1b5acb736ae48a315"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "d02431462a2cdc28200788ed8a7eef9d"
server_key_iv: "e8f5b3f33ec9038b44b383cd5e94e9cc"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.482005 27862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:54.503780 27862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:54.504029 27862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:54.504148 27862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:54.504325 27862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:54.504590 27862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:54.504621 27862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.504645 27862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:54.504658 27862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.510082 27862 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:43733
I20260806 22:43:54.510121 28065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:43733 every 8 connection(s)
I20260806 22:43:54.510411 27862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:54.515056 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.511057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:39203 (local address 127.0.186.254:34983)
0806 22:43:54.511102 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.511104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.511320 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:43:54.511799 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.511800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.511812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.511858 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:54.512494 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.512970 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.513914 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.514101 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.514146 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.514809 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:54.514810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:54.514810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:54.514817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:54.514830 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:54.514834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:54.514872 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.514956 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.514984 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:43:54.515218 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.511033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34983 (local address 127.0.186.193:39203)
0806 22:43:54.511209 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.511241 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.511720 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.511935 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.511939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.511947 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:54.512432 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.512436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.513047 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.513048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.513831 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.513833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.514172 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.514173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.514285 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.514692 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:54.514692 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:54.514719 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:54.514739 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:54.514901 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:54.514904 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:54.514907 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.514952 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.515030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 22:43:54.515722 28066 heartbeater.cc:347] Connected to a master server at 127.0.186.254:34983
I20260806 22:43:54.515810 28066 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:54.516009 28066 heartbeater.cc:511] Master 127.0.186.254:34983 requested a full tablet report, sending...
I20260806 22:43:54.516497 27771 ts_manager.cc:194] Registered new tserver with Master: 07aeb803f4624ab1b5acb736ae48a315 (127.0.186.193:43733)
I20260806 22:43:54.517086 27771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.193:39203
I20260806 22:43:54.520177 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 27862
I20260806 22:43:54.520272 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:54.520464 747 external_mini_cluster.cc:1428] Setting key fa0e1b6c0006f6020a2da2c7a054c5b7
I20260806 22:43:54.521270 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--builtin_ntp_servers=127.0.186.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:54.601739 28077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:54.601943 28077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:54.601974 28077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:54.603094 28077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:54.603132 28077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:54.603165 28077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:54.604543 28077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:54.604765 28077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:54.605032 28077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:54.609794 28091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.609807 28093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:54.609874 28077 server_base.cc:1034] running on GCE node
W20260806 22:43:54.609777 28090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:54.610143 28077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:54.610379 28077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:54.611515 28077 hybrid_clock.cc:648] HybridClock initialized: now 1786056234611490 us; error 45 us; skew 500 ppm
I20260806 22:43:54.612749 28077 webserver.cc:460] Webserver started at http://127.0.186.194:35109/ using document root <none> and password file <none>
I20260806 22:43:54.612977 28077 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:54.613027 28077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:54.613165 28077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:54.614208 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "1f60050820938fcc14b1a49e7a207693"
server_key_iv: "4c6948abbe7402019aaad0c90ea30abd"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.614552 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "1f60050820938fcc14b1a49e7a207693"
server_key_iv: "4c6948abbe7402019aaad0c90ea30abd"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.615808 28077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:54.616475 28103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.616638 28077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:54.616715 28077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "1f60050820938fcc14b1a49e7a207693"
server_key_iv: "4c6948abbe7402019aaad0c90ea30abd"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.616775 28077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:54.626751 28077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:54.627004 28077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:54.627125 28077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:54.627363 28077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:54.627663 28077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:54.627696 28077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.627722 28077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:54.627740 28077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.633210 28077 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:46507
I20260806 22:43:54.633249 28276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:46507 every 8 connection(s)
I20260806 22:43:54.633543 28077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:54.636300 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 28077
I20260806 22:43:54.636390 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:54.636562 747 external_mini_cluster.cc:1428] Setting key 354a2f220ab9a5e63e9b8eb4500a5cb9
I20260806 22:43:54.637440 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--builtin_ntp_servers=127.0.186.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:43:54.638724 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.634199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:39483 (local address 127.0.186.254:34983)
0806 22:43:54.634242 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.634244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.634470 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:43:54.634973 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.634974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.634991 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.635051 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:54.635751 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.636241 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.637396 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.637609 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.637674 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.638501 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:54.638503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:54.638503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:54.638507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:54.638524 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:54.638528 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:54.638566 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.638642 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.638668 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:43:54.638955 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.634181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34983 (local address 127.0.186.194:39483)
0806 22:43:54.634354 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.634387 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.634871 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.635119 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.635123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.635142 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:54.635661 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.635666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.636327 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.636328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.637323 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.637326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.637690 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.637691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.637846 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.638352 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:54.638353 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:54.638389 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:54.638415 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:54.638577 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:54.638581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:54.638584 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.638644 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.638751 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260806 22:43:54.639420 28277 heartbeater.cc:347] Connected to a master server at 127.0.186.254:34983
I20260806 22:43:54.639516 28277 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:54.639709 28277 heartbeater.cc:511] Master 127.0.186.254:34983 requested a full tablet report, sending...
I20260806 22:43:54.640076 27771 ts_manager.cc:194] Registered new tserver with Master: e3d7e04e1cda40bd8cfc4eddd7bb3105 (127.0.186.194:46507)
I20260806 22:43:54.640513 27771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.194:39483
W20260806 22:43:54.718534 28282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:54.718719 28282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:54.718739 28282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:54.719946 28282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:54.719990 28282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:54.720019 28282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:54.721431 28282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:54.721616 28282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:54.721853 28282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:43:54.726708 28282 server_base.cc:1034] running on GCE node
W20260806 22:43:54.726613 28289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.726675 28287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:54.726625 28293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:54.727017 28282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:54.727231 28282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:54.728401 28282 hybrid_clock.cc:648] HybridClock initialized: now 1786056234728391 us; error 27 us; skew 500 ppm
I20260806 22:43:54.729534 28282 webserver.cc:460] Webserver started at http://127.0.186.195:34467/ using document root <none> and password file <none>
I20260806 22:43:54.729758 28282 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:54.729809 28282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:54.729930 28282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:54.730911 28282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "df9fc8af1116456f9c22aba30680d92a"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "f9464f506303486e883ec96a36330362"
server_key_iv: "4b437fc504eae1da617fbeb80d68e57c"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.731238 28282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "df9fc8af1116456f9c22aba30680d92a"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "f9464f506303486e883ec96a36330362"
server_key_iv: "4b437fc504eae1da617fbeb80d68e57c"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.732448 28282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:43:54.733121 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.733266 28282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:54.733361 28282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "df9fc8af1116456f9c22aba30680d92a"
format_stamp: "Formatted at 2026-08-06 22:43:54 on dist-test-slave-gjrz"
server_key: "f9464f506303486e883ec96a36330362"
server_key_iv: "4b437fc504eae1da617fbeb80d68e57c"
server_key_version: "encryptionkey@0"
I20260806 22:43:54.733418 28282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:54.767174 28282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:54.767509 28282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:54.767642 28282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:54.767858 28282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:54.768172 28282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:54.768208 28282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.768237 28282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:54.768254 28282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:54.773615 28282 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:44375
I20260806 22:43:54.773674 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:44375 every 8 connection(s)
I20260806 22:43:54.773977 28282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:54.778741 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.774661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:41455 (local address 127.0.186.254:34983)
0806 22:43:54.774713 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.774715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.774954 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:43:54.775460 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.775461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.775471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.775526 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:54.776187 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.776662 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.777635 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.777827 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.777888 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.778556 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:54.778557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:54.778557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:54.778561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:54.778570 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:54.778573 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:54.778605 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.778665 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.778685 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:43:54.778914 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.774621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:34983 (local address 127.0.186.195:41455)
0806 22:43:54.774819 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.774858 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.775378 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.775592 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.775596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.775604 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:54.776109 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.776114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.776745 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.776746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.777537 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.777539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.777895 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.777896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.778019 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.778447 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:54.778447 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:54.778476 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:54.778498 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:54.778612 (+ 114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:54.778615 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:54.778617 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.778660 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.778735 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260806 22:43:54.779388 28492 heartbeater.cc:347] Connected to a master server at 127.0.186.254:34983
I20260806 22:43:54.779477 28492 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:54.779665 28492 heartbeater.cc:511] Master 127.0.186.254:34983 requested a full tablet report, sending...
I20260806 22:43:54.780001 27771 ts_manager.cc:194] Registered new tserver with Master: df9fc8af1116456f9c22aba30680d92a (127.0.186.195:44375)
I20260806 22:43:54.780359 27771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.195:41455
I20260806 22:43:54.783455 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 28282
I20260806 22:43:54.783546 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:54.783711 747 external_mini_cluster.cc:1428] Setting key d36c657a49296244a214e3401c192948
I20260806 22:43:54.784514 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:54.788321 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.785791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.0.186.254:34983)
0806 22:43:54.785841 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.785844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.785932 (+ 88us) server_negotiation.cc:407] Connection header received
0806 22:43:54.785985 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.785986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.785994 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.786051 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:54.786209 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.786649 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.787394 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.787574 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.787622 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.788135 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:54.788136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:54.788137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:54.788140 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:54.788152 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:54.788156 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:54.788195 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.788234 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.788253 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:43:54.789569 27771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:54.791052 27771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:54.796923 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:46507 (local address 127.0.0.1:50488)
0806 22:43:54.793213 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.793248 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.793309 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.793661 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.793664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.793667 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.793731 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.793733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.794733 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.794734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.795807 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.795810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.796641 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.796719 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.796732 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.796800 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.796851 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260806 22:43:54.796933 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.0.186.194:46507)
0806 22:43:54.793497 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.793503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.793514 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:54.793554 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.793556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.793572 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.793651 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.793809 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.794657 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.795901 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.796568 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796627 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.796725 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.796816 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.796854 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260806 22:43:54.797096 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:43733 (local address 127.0.0.1:57834)
0806 22:43:54.793483 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.793493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.793545 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.793767 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.793768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.793770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.793816 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.793818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.795150 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.795151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.795956 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.795959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.796858 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.796924 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.796927 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.796986 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.797026 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":95,"threads_started":1}
I20260806 22:43:54.797300 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44375 (local address 127.0.0.1:48382)
0806 22:43:54.793657 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.793667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.793708 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.793997 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.793998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.794000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.794034 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.794035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.795215 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.795216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.796275 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.796277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.797083 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.797084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.797145 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.797149 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.797205 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.797240 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":68,"threads_started":1}
I20260806 22:43:54.797328 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.0.186.195:44375)
0806 22:43:54.793809 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.793813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.793821 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:43:54.793852 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.793854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.793868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.793936 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.794104 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.795138 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796355 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.797021 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.797081 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.797158 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.797223 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.797257 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":72,"threads_started":1}
I20260806 22:43:54.797361 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.793318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.0.186.193:43733)
0806 22:43:54.793525 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.793529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.793538 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:54.793617 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.793619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.793632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.793699 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.793916 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.795053 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796057 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.796668 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.796739 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.796820 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.797247 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.797285 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260806 22:43:54.798024 28390 tablet_service.cc:1467] Processing CreateTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d (DEFAULT_TABLE table=test-table [id=45557ce39d104a8da3e7a64bfe89c481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:54.798067 27963 tablet_service.cc:1467] Processing CreateTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d (DEFAULT_TABLE table=test-table [id=45557ce39d104a8da3e7a64bfe89c481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:54.798373 28390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eeda0d3392c425e8f59c46a45ab183d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:54.798429 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eeda0d3392c425e8f59c46a45ab183d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:54.798491 28173 tablet_service.cc:1467] Processing CreateTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d (DEFAULT_TABLE table=test-table [id=45557ce39d104a8da3e7a64bfe89c481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:54.798794 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eeda0d3392c425e8f59c46a45ab183d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:54.800927 28520 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: Bootstrap starting.
I20260806 22:43:54.801623 28520 tablet_bootstrap.cc:654] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:54.801601 28521 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Bootstrap starting.
I20260806 22:43:54.801959 28520 log.cc:824] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:54.802248 28521 tablet_bootstrap.cc:654] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:54.802280 28522 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Bootstrap starting.
I20260806 22:43:54.802572 28521 log.cc:824] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:54.802757 28520 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: No bootstrap required, opened a new log
I20260806 22:43:54.802831 28520 ts_tablet_manager.cc:1397] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:54.802886 28522 tablet_bootstrap.cc:654] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:54.803172 28522 log.cc:824] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:54.803440 28521 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: No bootstrap required, opened a new log
I20260806 22:43:54.803516 28521 ts_tablet_manager.cc:1397] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:54.803803 28522 tablet_bootstrap.cc:492] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: No bootstrap required, opened a new log
I20260806 22:43:54.803861 28522 ts_tablet_manager.cc:1397] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:54.805326 28520 raft_consensus.cc:348] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.805508 28520 raft_consensus.cc:374] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:54.805544 28520 raft_consensus.cc:729] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07aeb803f4624ab1b5acb736ae48a315, State: Initialized, Role: FOLLOWER
I20260806 22:43:54.805675 28520 consensus_queue.cc:260] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.805714 28522 raft_consensus.cc:348] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.805853 28522 raft_consensus.cc:374] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:54.805895 28522 raft_consensus.cc:729] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d7e04e1cda40bd8cfc4eddd7bb3105, State: Initialized, Role: FOLLOWER
I20260806 22:43:54.805941 28520 ts_tablet_manager.cc:1428] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:54.806074 28066 heartbeater.cc:503] Master 127.0.186.254:34983 was elected leader, sending a full tablet report...
I20260806 22:43:54.806071 28522 consensus_queue.cc:260] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.806183 28530 raft_consensus.cc:482] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:54.806152 28521 raft_consensus.cc:348] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.806295 28521 raft_consensus.cc:374] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:54.806306 28522 ts_tablet_manager.cc:1428] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:43:54.806337 28521 raft_consensus.cc:729] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9fc8af1116456f9c22aba30680d92a, State: Initialized, Role: FOLLOWER
I20260806 22:43:54.806412 28277 heartbeater.cc:503] Master 127.0.186.254:34983 was elected leader, sending a full tablet report...
I20260806 22:43:54.806445 28521 consensus_queue.cc:260] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.806249 28530 raft_consensus.cc:504] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.806690 28521 ts_tablet_manager.cc:1428] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:54.806732 28492 heartbeater.cc:503] Master 127.0.186.254:34983 was elected leader, sending a full tablet report...
I20260806 22:43:54.806882 28530 leader_election.cc:288] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07aeb803f4624ab1b5acb736ae48a315 (127.0.186.193:43733), df9fc8af1116456f9c22aba30680d92a (127.0.186.195:44375)
I20260806 22:43:54.810402 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.807020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:43733 (local address 127.0.186.194:59905)
0806 22:43:54.807328 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.807338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.807376 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.807703 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.807704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.807705 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.807761 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.807763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.808575 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.808575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.809631 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.809633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.810235 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.810236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.810285 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.810289 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.810326 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.810356 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290}
I20260806 22:43:54.810428 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.807337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:59905 (local address 127.0.186.193:43733)
0806 22:43:54.807529 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.807530 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.807537 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:43:54.807557 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.807558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.807567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.807610 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.807837 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.808503 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.809720 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.810163 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.810242 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.810249 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.810331 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.810361 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260806 22:43:54.810678 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eeda0d3392c425e8f59c46a45ab183d" candidate_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aeb803f4624ab1b5acb736ae48a315" is_pre_election: true
I20260806 22:43:54.810804 27997 raft_consensus.cc:2454] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d7e04e1cda40bd8cfc4eddd7bb3105 in term 0.
I20260806 22:43:54.810858 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.807106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:44375 (local address 127.0.186.194:40501)
0806 22:43:54.807869 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.807877 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.807921 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.808052 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:54.808053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:54.808055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:54.808110 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.808111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.808839 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.808840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.809855 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:54.809857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.810653 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.810654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:54.810711 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.810714 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:54.810753 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 22:43:54.810788 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":76,"threads_started":1}
I20260806 22:43:54.810870 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.807662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:40501 (local address 127.0.186.195:44375)
0806 22:43:54.807696 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.807698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.807925 (+ 227us) server_negotiation.cc:407] Connection header received
0806 22:43:54.807967 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.807968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.807976 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.808017 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:54.808174 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.808774 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.809928 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.810436 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.810499 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.810506 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.810778 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.810818 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:43:54.811035 28114 leader_election.cc:302] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aeb803f4624ab1b5acb736ae48a315, e3d7e04e1cda40bd8cfc4eddd7bb3105; no voters:
I20260806 22:43:54.811165 28532 raft_consensus.cc:2781] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:54.811245 28532 raft_consensus.cc:482] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:54.811303 28532 raft_consensus.cc:3037] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.811336 28416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eeda0d3392c425e8f59c46a45ab183d" candidate_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9fc8af1116456f9c22aba30680d92a" is_pre_election: true
I20260806 22:43:54.811450 28416 raft_consensus.cc:2454] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d7e04e1cda40bd8cfc4eddd7bb3105 in term 0.
I20260806 22:43:54.811991 28532 raft_consensus.cc:504] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.812093 28532 leader_election.cc:288] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [CANDIDATE]: Term 1 election: Requested vote from peers 07aeb803f4624ab1b5acb736ae48a315 (127.0.186.193:43733), df9fc8af1116456f9c22aba30680d92a (127.0.186.195:44375)
I20260806 22:43:54.812268 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eeda0d3392c425e8f59c46a45ab183d" candidate_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aeb803f4624ab1b5acb736ae48a315"
I20260806 22:43:54.812270 28416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eeda0d3392c425e8f59c46a45ab183d" candidate_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9fc8af1116456f9c22aba30680d92a"
I20260806 22:43:54.812336 28416 raft_consensus.cc:3037] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.812337 27997 raft_consensus.cc:3037] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:54.812979 28416 raft_consensus.cc:2454] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d7e04e1cda40bd8cfc4eddd7bb3105 in term 1.
I20260806 22:43:54.812978 27997 raft_consensus.cc:2454] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d7e04e1cda40bd8cfc4eddd7bb3105 in term 1.
I20260806 22:43:54.813122 28114 leader_election.cc:302] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aeb803f4624ab1b5acb736ae48a315, e3d7e04e1cda40bd8cfc4eddd7bb3105; no voters:
I20260806 22:43:54.813205 28532 raft_consensus.cc:2781] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:54.813274 28532 raft_consensus.cc:686] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 1 LEADER]: Becoming Leader. State: Replica: e3d7e04e1cda40bd8cfc4eddd7bb3105, State: Running, Role: LEADER
I20260806 22:43:54.813347 28532 consensus_queue.cc:237] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } }
I20260806 22:43:54.814167 27767 catalog_manager.cc:5495] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3d7e04e1cda40bd8cfc4eddd7bb3105 (127.0.186.194). New cstate: current_term: 1 leader_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3d7e04e1cda40bd8cfc4eddd7bb3105" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 46507 } health_report { overall_health: HEALTHY } } }
I20260806 22:43:54.824344 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:54.822229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.0.186.194:46507)
0806 22:43:54.822277 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:54.822279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:54.822285 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:43:54.822358 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:54.822358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:54.822372 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:54.822424 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:54.822515 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.822948 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.823688 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:54.823874 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:54.823935 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:54.824077 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:54.824165 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:54.824228 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:54.824264 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:43:54.824279 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:43:54.825855 27997 raft_consensus.cc:1261] T 3eeda0d3392c425e8f59c46a45ab183d P 07aeb803f4624ab1b5acb736ae48a315 [term 1 FOLLOWER]: Refusing update from remote peer e3d7e04e1cda40bd8cfc4eddd7bb3105: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:54.825874 28416 raft_consensus.cc:1261] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 1 FOLLOWER]: Refusing update from remote peer e3d7e04e1cda40bd8cfc4eddd7bb3105: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:54.826179 28530 consensus_queue.cc:1035] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df9fc8af1116456f9c22aba30680d92a" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 44375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:54.826285 28532 consensus_queue.cc:1035] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07aeb803f4624ab1b5acb736ae48a315" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 43733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:54.828760 28546 mvcc.cc:205] Tried to move back new op lower bound from 7315686337844539392 to 7315686337796182016. Current Snapshot: MvccSnapshot[applied={T|T < 7315686337844539392}]
I20260806 22:43:54.829071 28543 mvcc.cc:205] Tried to move back new op lower bound from 7315686337844539392 to 7315686337796182016. Current Snapshot: MvccSnapshot[applied={T|T < 7315686337844539392}]
I20260806 22:43:54.831034 27771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:54.831110 27771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:54.831868 27771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbf82a58d32e4579813fa98af7b36849: Sending DeleteTablet for 3 replicas of tablet 3eeda0d3392c425e8f59c46a45ab183d
I20260806 22:43:54.832203 27963 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='slave'} at 127.0.0.1:57834
I20260806 22:43:54.832248 28173 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='slave'} at 127.0.0.1:50488
I20260806 22:43:54.832253 28390 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eeda0d3392c425e8f59c46a45ab183d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:54 UTC) from {username='slave'} at 127.0.0.1:48382
I20260806 22:43:54.832466 28564 tablet_replica.cc:330] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: stopping tablet replica
I20260806 22:43:54.832579 28564 raft_consensus.cc:2229] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:54.832618 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 27862
I20260806 22:43:54.832685 28564 raft_consensus.cc:2258] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:54.843441 28563 tablet_replica.cc:330] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: stopping tablet replica
I20260806 22:43:54.843576 28563 raft_consensus.cc:2229] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:43:54.843724 28563 raft_consensus.cc:2258] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:54.844277 28564 ts_tablet_manager.cc:1905] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:54.844277 28563 ts_tablet_manager.cc:1905] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:54.846097 28563 ts_tablet_manager.cc:1918] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:54.846097 28564 ts_tablet_manager.cc:1918] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:54.846177 28564 log.cc:1196] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/3eeda0d3392c425e8f59c46a45ab183d
I20260806 22:43:54.846283 28563 log.cc:1196] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/3eeda0d3392c425e8f59c46a45ab183d
I20260806 22:43:54.846431 28564 ts_tablet_manager.cc:1939] T 3eeda0d3392c425e8f59c46a45ab183d P df9fc8af1116456f9c22aba30680d92a: Deleting consensus metadata
I20260806 22:43:54.846657 28563 ts_tablet_manager.cc:1939] T 3eeda0d3392c425e8f59c46a45ab183d P e3d7e04e1cda40bd8cfc4eddd7bb3105: Deleting consensus metadata
W20260806 22:43:54.846784 27744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:43733 (error 108)
I20260806 22:43:54.847018 27744 catalog_manager.cc:4841] TS df9fc8af1116456f9c22aba30680d92a (127.0.186.195:44375): tablet 3eeda0d3392c425e8f59c46a45ab183d (table test-table [id=45557ce39d104a8da3e7a64bfe89c481]) successfully deleted
I20260806 22:43:54.847136 27738 catalog_manager.cc:4841] TS e3d7e04e1cda40bd8cfc4eddd7bb3105 (127.0.186.194:46507): tablet 3eeda0d3392c425e8f59c46a45ab183d (table test-table [id=45557ce39d104a8da3e7a64bfe89c481]) successfully deleted
W20260806 22:43:54.847412 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:43:54.847195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:43733 (local address 127.0.0.1:57848)
0806 22:43:54.847247 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:54.847258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:54.847326 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:54.847368 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:43733: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:43:54.847532 27744 catalog_manager.cc:4584] TS 07aeb803f4624ab1b5acb736ae48a315 (127.0.186.193:43733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3eeda0d3392c425e8f59c46a45ab183d: Network error: Client connection negotiation failed: client connection to 127.0.186.193:43733: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 22:43:54.847648 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 28077
I20260806 22:43:54.860277 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 28282
I20260806 22:43:54.872062 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 27713
2026-08-06T22:43:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (660 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:54 dist-test-slave-gjrz krb5kdc[28572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:54 dist-test-slave-gjrz krb5kdc[28572](info): set up 2 sockets
Aug 06 22:43:54 dist-test-slave-gjrz krb5kdc[28572](info): commencing operation
krb5kdc: starting...
W20260806 22:43:56.000883 747 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:43:56.115707 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35131
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:35131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:56.197036 28597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:56.197223 28597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:56.197253 28597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:56.198365 28597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:56.198407 28597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:56.198426 28597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:56.198442 28597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:56.198458 28597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:56.200021 28597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:35131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:56.200259 28597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:56.200461 28597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:56.204974 28604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.205024 28603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.205128 28597 server_base.cc:1034] running on GCE node
W20260806 22:43:56.205173 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.205421 28597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:56.205689 28597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:56.206816 28597 hybrid_clock.cc:648] HybridClock initialized: now 1786056236206800 us; error 31 us; skew 500 ppm
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:56.209153 28597 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:43:56.209527 28597 webserver.cc:460] Webserver started at http://127.0.186.254:38627/ using document root <none> and password file <none>
I20260806 22:43:56.209713 28597 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:56.209769 28597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:56.209882 28597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:56.210862 28597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "bb4a583e44d84471ad9ae32e2603bd27"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "fa8c48780f82d901afcf59ce0e31b23d"
server_key_iv: "cb19231c02f05cc3de2f616f5bd1ae6b"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.211181 28597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "bb4a583e44d84471ad9ae32e2603bd27"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "fa8c48780f82d901afcf59ce0e31b23d"
server_key_iv: "cb19231c02f05cc3de2f616f5bd1ae6b"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.212471 28597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:56.213229 28624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.213404 28597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:56.213486 28597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "bb4a583e44d84471ad9ae32e2603bd27"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "fa8c48780f82d901afcf59ce0e31b23d"
server_key_iv: "cb19231c02f05cc3de2f616f5bd1ae6b"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.213549 28597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:56.236073 28597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:56.237874 28597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:56.237979 28597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:56.241680 28597 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:35131
I20260806 22:43:56.241722 28707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:35131 every 8 connection(s)
I20260806 22:43:56.242041 28597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:56.242913 28708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:56.245206 28708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: Bootstrap starting.
I20260806 22:43:56.245759 28708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:56.246038 28708 log.cc:824] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:56.246680 28708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: No bootstrap required, opened a new log
I20260806 22:43:56.248275 28708 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } }
I20260806 22:43:56.248414 28708 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:56.248436 28708 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4a583e44d84471ad9ae32e2603bd27, State: Initialized, Role: FOLLOWER
I20260806 22:43:56.248564 28708 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } }
I20260806 22:43:56.248631 28708 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:56.248651 28708 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:56.248682 28708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:56.249184 28708 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } }
I20260806 22:43:56.249276 28708 leader_election.cc:302] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4a583e44d84471ad9ae32e2603bd27; no voters:
I20260806 22:43:56.249428 28708 leader_election.cc:288] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:56.249464 28712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:56.249635 28712 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [term 1 LEADER]: Becoming Leader. State: Replica: bb4a583e44d84471ad9ae32e2603bd27, State: Running, Role: LEADER
I20260806 22:43:56.249673 28708 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:56.249725 28712 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } }
I20260806 22:43:56.250089 28716 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4a583e44d84471ad9ae32e2603bd27. Latest consensus state: current_term: 1 leader_uuid: "bb4a583e44d84471ad9ae32e2603bd27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } } }
I20260806 22:43:56.250177 28716 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:56.250209 28714 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb4a583e44d84471ad9ae32e2603bd27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a583e44d84471ad9ae32e2603bd27" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35131 } } }
I20260806 22:43:56.250283 28714 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:56.250440 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 28597
I20260806 22:43:56.250504 28725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:56.250546 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:56.250746 747 external_mini_cluster.cc:1428] Setting key d0a6625225a8f32b85e573e4241b9817
I20260806 22:43:56.251020 28725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:56.252710 28725 catalog_manager.cc:1347] Generated new cluster ID: 99e6cb9a5e7748ab8135b059e865b83a
I20260806 22:43:56.252761 28725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:56.260857 28725 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:56.261441 28725 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:56.262279 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.251444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.0.186.254:35131)
0806 22:43:56.251670 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.251678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.251711 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:43:56.251827 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.251835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.251892 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.252020 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:56.252775 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.254283 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.255327 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.256000 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.256129 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.258788 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:56.258793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:56.258793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:56.258804 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:56.260654 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.261103 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.261106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.261106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.261207 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.261397 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.261398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.261398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.261661 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:56.261809 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.261920 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.262053 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260806 22:43:56.268469 28725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: Generated new TSK 0
I20260806 22:43:56.268633 28725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:43:56.293574 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:56.374295 28758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:56.374493 28758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:56.374516 28758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:56.375666 28758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:56.375705 28758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:56.375730 28758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:56.377170 28758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:56.377374 28758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:56.377580 28758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:56.382381 28768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.382421 28770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.382578 28758 server_base.cc:1034] running on GCE node
W20260806 22:43:56.382625 28774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.382835 28758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:56.383078 28758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:56.384240 28758 hybrid_clock.cc:648] HybridClock initialized: now 1786056236384225 us; error 26 us; skew 500 ppm
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:56.386775 28758 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:43:56.387210 28758 webserver.cc:460] Webserver started at http://127.0.186.193:43025/ using document root <none> and password file <none>
I20260806 22:43:56.387440 28758 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:56.387491 28758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:56.387612 28758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:56.388628 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "f7d70991ed2f4a0caf6107788245484a"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "cf2a4b36b8fce78a572ccbd5d1c055f2"
server_key_iv: "af6c9bdfa6dd8891f55f213345e98483"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.388957 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "f7d70991ed2f4a0caf6107788245484a"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "cf2a4b36b8fce78a572ccbd5d1c055f2"
server_key_iv: "af6c9bdfa6dd8891f55f213345e98483"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.390193 28758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:56.390931 28783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.391093 28758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:56.391183 28758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "f7d70991ed2f4a0caf6107788245484a"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "cf2a4b36b8fce78a572ccbd5d1c055f2"
server_key_iv: "af6c9bdfa6dd8891f55f213345e98483"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.391242 28758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:56.413628 28758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:56.415997 28758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:56.416145 28758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:56.416399 28758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:56.416736 28758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:56.416777 28758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.416815 28758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:56.416833 28758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.424047 28758 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:38029
I20260806 22:43:56.424094 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:38029 every 8 connection(s)
I20260806 22:43:56.424476 28758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:56.428586 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 28758
I20260806 22:43:56.428707 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:56.428907 747 external_mini_cluster.cc:1428] Setting key e500611c92d6cda07d06e1fffbea7fd8
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:56.435575 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.425206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:39027 (local address 127.0.186.254:35131)
0806 22:43:56.425268 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.425271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.425456 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:43:56.426377 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.426378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.426398 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.426476 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:56.427100 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.427988 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.429391 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.429667 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.429726 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.432374 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:56.432376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:56.432376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:56.432385 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:56.433957 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.434453 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.434454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.434455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.434528 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.434806 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.434807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.434807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.434943 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:56.435012 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.435369 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.435478 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:43:56.435774 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.425164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35131 (local address 127.0.186.193:39027)
0806 22:43:56.425334 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.425373 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.426277 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.426535 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.426538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.426794 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:56.427019 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.427025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.428105 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.428106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.429316 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.429319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.429761 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.429762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.429928 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.430445 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:56.430456 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:56.432284 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:56.434055 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.434064 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.434067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.434370 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.434600 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.434601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.434602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.434738 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.435012 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:56.435020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:56.435187 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.435347 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.435542 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260806 22:43:56.436239 28968 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35131
I20260806 22:43:56.436345 28968 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:56.436543 28968 heartbeater.cc:511] Master 127.0.186.254:35131 requested a full tablet report, sending...
I20260806 22:43:56.437116 28656 ts_manager.cc:194] Registered new tserver with Master: f7d70991ed2f4a0caf6107788245484a (127.0.186.193:38029)
I20260806 22:43:56.438020 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:39027
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:43:56.454051 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:56.533814 28975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:56.534008 28975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:56.534030 28975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:56.535149 28975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:56.535189 28975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:56.535213 28975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:56.536706 28975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:56.536918 28975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:56.537124 28975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:56.541953 28991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.542120 28975 server_base.cc:1034] running on GCE node
W20260806 22:43:56.541988 28987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.541961 28988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.542460 28975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:56.542657 28975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:56.543797 28975 hybrid_clock.cc:648] HybridClock initialized: now 1786056236543781 us; error 29 us; skew 500 ppm
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:56.546234 28975 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:43:56.546692 28975 webserver.cc:460] Webserver started at http://127.0.186.194:34125/ using document root <none> and password file <none>
I20260806 22:43:56.546895 28975 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:56.546943 28975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:56.547070 28975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:56.548130 28975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "44eb26f579794a689c1e4e589e41f946"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "b80808edf21fbeedea446bc6bd604c1d"
server_key_iv: "e185906fb9670ed82a23548a65bd4a1e"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.548481 28975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "44eb26f579794a689c1e4e589e41f946"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "b80808edf21fbeedea446bc6bd604c1d"
server_key_iv: "e185906fb9670ed82a23548a65bd4a1e"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.549729 28975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:43:56.550400 29000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.550575 28975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:56.550673 28975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "44eb26f579794a689c1e4e589e41f946"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "b80808edf21fbeedea446bc6bd604c1d"
server_key_iv: "e185906fb9670ed82a23548a65bd4a1e"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.550760 28975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:56.573688 28975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:56.575804 28975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:56.575975 28975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:56.576198 28975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:56.576514 28975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:56.576548 28975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.576576 28975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:56.576604 28975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.582286 28975 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:45595
I20260806 22:43:56.582350 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:45595 every 8 connection(s)
I20260806 22:43:56.582643 28975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:56.589026 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 28975
I20260806 22:43:56.589128 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:56.589314 747 external_mini_cluster.cc:1428] Setting key 922222c7d83594c7c06e41ec974a6637
I20260806 22:43:56.591732 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.583330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:57753 (local address 127.0.186.254:35131)
0806 22:43:56.583385 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.583387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.583608 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:43:56.584519 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.584520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.584532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.584591 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:56.585188 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.585691 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.586696 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.586890 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.586956 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.589015 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:56.589016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:56.589017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:56.589023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:56.590493 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.590851 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.590852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.590852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.590916 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.591119 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.591120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.591120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.591239 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:56.591333 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.591571 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.591647 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:43:56.591867 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.583310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35131 (local address 127.0.186.194:57753)
0806 22:43:56.583481 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.583522 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.584405 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.584660 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.584664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.584909 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:56.585102 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.585106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.585768 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.585769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.586592 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.586596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.586963 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.586963 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.587056 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.587541 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:56.587549 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:56.588921 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:56.590570 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.590577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.590579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.590787 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.590972 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.590973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.590973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.591067 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.591326 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:56.591331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:56.591458 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.591539 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.591655 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260806 22:43:56.592309 29193 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35131
I20260806 22:43:56.592410 29193 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:56.592599 29193 heartbeater.cc:511] Master 127.0.186.254:35131 requested a full tablet report, sending...
I20260806 22:43:56.592980 28656 ts_manager.cc:194] Registered new tserver with Master: 44eb26f579794a689c1e4e589e41f946 (127.0.186.194:45595)
I20260806 22:43:56.593334 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:57753
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:43:56.613917 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:56.695034 29202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:56.695247 29202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:56.695292 29202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:56.696411 29202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:56.696456 29202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:56.696481 29202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:56.697906 29202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:56.698082 29202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:56.698261 29202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:56.702852 29216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.702898 29214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.703060 29218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.703040 29202 server_base.cc:1034] running on GCE node
I20260806 22:43:56.703299 29202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:56.703557 29202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:56.704689 29202 hybrid_clock.cc:648] HybridClock initialized: now 1786056236704677 us; error 26 us; skew 500 ppm
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:56.707082 29202 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:43:56.707530 29202 webserver.cc:460] Webserver started at http://127.0.186.195:35581/ using document root <none> and password file <none>
I20260806 22:43:56.707718 29202 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:56.707762 29202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:56.707878 29202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:56.708822 29202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "3233312369db4200b49bd8d270d0f44d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "eb62ab2932451810dbe0432bc78874c6"
server_key_iv: "ca2259e399846db0c9fc464cb9c0d30d"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.709131 29202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "3233312369db4200b49bd8d270d0f44d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "eb62ab2932451810dbe0432bc78874c6"
server_key_iv: "ca2259e399846db0c9fc464cb9c0d30d"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.710342 29202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:56.711007 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.711158 29202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:56.711232 29202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "3233312369db4200b49bd8d270d0f44d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "eb62ab2932451810dbe0432bc78874c6"
server_key_iv: "ca2259e399846db0c9fc464cb9c0d30d"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.711329 29202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:56.723871 29202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:56.725831 29202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:56.725946 29202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:56.726141 29202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:56.726418 29202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:56.726452 29202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.726492 29202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:56.726517 29202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.732059 29202 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45899
I20260806 22:43:56.732118 29433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45899 every 8 connection(s)
I20260806 22:43:56.732384 29202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:43:56 dist-test-slave-gjrz krb5kdc[28572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:56.739027 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 29202
I20260806 22:43:56.739142 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:56.739355 747 external_mini_cluster.cc:1428] Setting key c1488103186f323af1ca6901eda25eec
I20260806 22:43:56.740940 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.733104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:41055 (local address 127.0.186.254:35131)
0806 22:43:56.733160 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.733162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.733365 (+ 203us) server_negotiation.cc:407] Connection header received
0806 22:43:56.734218 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.734219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.734230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.734277 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:56.734819 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.735338 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.736256 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.736431 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.736474 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.738342 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:56.738343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:56.738343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:56.738349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:56.739800 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.740234 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.740234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.740235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.740297 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.740495 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.740496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.740496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.740590 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:56.740634 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.740820 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.740879 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:43:56.741093 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.733101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35131 (local address 127.0.186.195:41055)
0806 22:43:56.733257 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.733290 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.734130 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.734340 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.734343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.734583 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:56.734761 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.734765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.735397 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.735397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.736184 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.736186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.736489 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.736489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.736580 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.737000 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:56.737008 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:56.738264 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:56.739875 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.739881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.739884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.740168 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.740352 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:56.740353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:56.740353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:56.740446 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:56.740624 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:56.740627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:56.740730 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.740801 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.740914 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260806 22:43:56.741497 29434 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35131
I20260806 22:43:56.741595 29434 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:56.741757 29434 heartbeater.cc:511] Master 127.0.186.254:35131 requested a full tablet report, sending...
I20260806 22:43:56.742081 28656 ts_manager.cc:194] Registered new tserver with Master: 3233312369db4200b49bd8d270d0f44d (127.0.186.195:45899)
I20260806 22:43:56.742424 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:41055
I20260806 22:43:56.750936 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:56.758024 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.752295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.0.186.254:35131)
0806 22:43:56.752345 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.752347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.752457 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:43:56.752521 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.752521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.752531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.752577 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:56.753106 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.753616 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.754418 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.754612 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.754664 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.755890 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:56.755891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:56.755891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:56.755897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:56.756991 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.757336 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.757336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.757337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.757384 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:56.757542 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:56.757542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:56.757543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:56.757658 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:56.757733 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.757904 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.757953 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:43:56.759539 28656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:56.761044 28656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:56.767387 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.0.186.194:45595)
0806 22:43:56.763824 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.763828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.763837 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:56.763872 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.763873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.763889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.763957 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:56.764185 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.765243 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.766282 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.767005 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.767077 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.767140 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.767247 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.767308 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 22:43:56.767379 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:45595 (local address 127.0.0.1:40728)
0806 22:43:56.763650 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.763658 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.763693 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.764071 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.764072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.764074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:56.764124 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.764125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.765324 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.765325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.766209 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.766212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.767092 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.767093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.767152 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.767166 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.767232 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.767306 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260806 22:43:56.767582 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.0.186.193:38029)
0806 22:43:56.763498 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.763502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.763546 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:43:56.763645 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.763648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.763664 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.763743 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:56.764012 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.765153 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.766422 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.767128 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.767189 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.767237 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.767446 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.767491 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260806 22:43:56.767714 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:38029 (local address 127.0.0.1:55080)
0806 22:43:56.763443 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.763489 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.763562 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.763758 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.763761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.763763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:56.763833 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.763835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.765256 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.765257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.766341 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.766343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.767204 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.767205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.767290 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.767294 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.767340 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.767376 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260806 22:43:56.768265 29086 tablet_service.cc:1467] Processing CreateTablet for tablet 3c312ef1744e4142b28d43b52dc968be (DEFAULT_TABLE table=test-table [id=8577f70263fc4d43883e0e2cfbb13beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:56.768369 28855 tablet_service.cc:1467] Processing CreateTablet for tablet 3c312ef1744e4142b28d43b52dc968be (DEFAULT_TABLE table=test-table [id=8577f70263fc4d43883e0e2cfbb13beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:56.768653 29086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c312ef1744e4142b28d43b52dc968be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:56.768657 28855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c312ef1744e4142b28d43b52dc968be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:56.771986 29459 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: Bootstrap starting.
I20260806 22:43:56.772078 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.0.186.195:45899)
0806 22:43:56.763932 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.763936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.763946 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:43:56.763983 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.763985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.764001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.764070 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:56.764320 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.764984 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.767293 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.771701 (+ 4408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.771768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.771823 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.771956 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.772003 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":99,"threads_started":1}
I20260806 22:43:56.772277 29460 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Bootstrap starting.
I20260806 22:43:56.772405 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.763185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45899 (local address 127.0.0.1:56092)
0806 22:43:56.763509 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.763521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.763574 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.764204 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.764205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.764207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:56.764253 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.764254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.765081 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.765082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.766342 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.766343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.771782 (+ 5439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.771783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.771856 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.771860 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.772289 (+ 429us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.772342 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260806 22:43:56.772850 29460 tablet_bootstrap.cc:654] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:56.773047 29316 tablet_service.cc:1467] Processing CreateTablet for tablet 3c312ef1744e4142b28d43b52dc968be (DEFAULT_TABLE table=test-table [id=8577f70263fc4d43883e0e2cfbb13beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:56.773310 29460 log.cc:824] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:56.773339 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c312ef1744e4142b28d43b52dc968be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:56.774061 29459 tablet_bootstrap.cc:654] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:56.774124 29460 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: No bootstrap required, opened a new log
I20260806 22:43:56.774196 29460 ts_tablet_manager.cc:1397] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:56.774657 29459 log.cc:824] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:56.775548 29459 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: No bootstrap required, opened a new log
I20260806 22:43:56.775620 29459 ts_tablet_manager.cc:1397] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:43:56.776119 29460 raft_consensus.cc:348] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.776228 29466 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Bootstrap starting.
I20260806 22:43:56.776300 29460 raft_consensus.cc:374] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:56.776342 29460 raft_consensus.cc:729] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44eb26f579794a689c1e4e589e41f946, State: Initialized, Role: FOLLOWER
I20260806 22:43:56.776459 29460 consensus_queue.cc:260] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.776760 29460 ts_tablet_manager.cc:1428] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:43:56.776906 29193 heartbeater.cc:503] Master 127.0.186.254:35131 was elected leader, sending a full tablet report...
I20260806 22:43:56.777254 29466 tablet_bootstrap.cc:654] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:56.777459 29459 raft_consensus.cc:348] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.777634 29459 raft_consensus.cc:374] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:56.777686 29459 raft_consensus.cc:729] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d70991ed2f4a0caf6107788245484a, State: Initialized, Role: FOLLOWER
I20260806 22:43:56.777756 29466 log.cc:824] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:56.777812 29459 consensus_queue.cc:260] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.778074 29459 ts_tablet_manager.cc:1428] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:43:56.778091 28968 heartbeater.cc:503] Master 127.0.186.254:35131 was elected leader, sending a full tablet report...
I20260806 22:43:56.778774 29466 tablet_bootstrap.cc:492] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: No bootstrap required, opened a new log
I20260806 22:43:56.778843 29466 ts_tablet_manager.cc:1397] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:43:56.780263 29466 raft_consensus.cc:348] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.780392 29466 raft_consensus.cc:374] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:56.780423 29466 raft_consensus.cc:729] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3233312369db4200b49bd8d270d0f44d, State: Initialized, Role: FOLLOWER
I20260806 22:43:56.780522 29466 consensus_queue.cc:260] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.780764 29466 ts_tablet_manager.cc:1428] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:56.780781 29434 heartbeater.cc:503] Master 127.0.186.254:35131 was elected leader, sending a full tablet report...
I20260806 22:43:56.797421 29468 raft_consensus.cc:482] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:56.797521 29468 raft_consensus.cc:504] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.797833 29468 leader_election.cc:288] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44eb26f579794a689c1e4e589e41f946 (127.0.186.194:45595), 3233312369db4200b49bd8d270d0f44d (127.0.186.195:45899)
I20260806 22:43:56.800830 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.798064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:58501 (local address 127.0.186.194:45595)
0806 22:43:56.798105 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.798107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.798129 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:43:56.798153 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.798154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.798163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.798205 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:56.798367 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.798878 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.799981 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.800518 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.800571 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.800578 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.800747 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.800782 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:43:56.800837 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.797974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:45595 (local address 127.0.186.193:58501)
0806 22:43:56.798024 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.798037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.798086 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.798217 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.798218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.798219 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:56.798294 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.798296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.798949 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.798950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.799867 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.799870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.800590 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.800591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.800673 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.800676 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.800729 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.800768 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:43:56.801142 29117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c312ef1744e4142b28d43b52dc968be" candidate_uuid: "f7d70991ed2f4a0caf6107788245484a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44eb26f579794a689c1e4e589e41f946" is_pre_election: true
I20260806 22:43:56.801223 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.797974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45899 (local address 127.0.186.193:52119)
0806 22:43:56.798399 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.798407 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:56.798450 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:56.798573 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:56.798574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:56.798575 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:56.798636 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.798637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.799434 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.799435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.800338 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:56.800341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.801004 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.801005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:56.801066 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.801070 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:56.801123 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:43:56.801159 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":264,"thread_start_us":98,"threads_started":1}
I20260806 22:43:56.801223 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.798044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:52119 (local address 127.0.186.195:45899)
0806 22:43:56.798083 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.798084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.798455 (+ 371us) server_negotiation.cc:407] Connection header received
0806 22:43:56.798509 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.798510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.798527 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.798572 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:56.798690 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.799345 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.800431 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.800913 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.800987 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.800996 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.801143 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.801180 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:43:56.801314 29117 raft_consensus.cc:2454] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7d70991ed2f4a0caf6107788245484a in term 0.
I20260806 22:43:56.801522 28788 leader_election.cc:302] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44eb26f579794a689c1e4e589e41f946, f7d70991ed2f4a0caf6107788245484a; no voters:
I20260806 22:43:56.801504 29358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c312ef1744e4142b28d43b52dc968be" candidate_uuid: "f7d70991ed2f4a0caf6107788245484a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3233312369db4200b49bd8d270d0f44d" is_pre_election: true
I20260806 22:43:56.801637 29358 raft_consensus.cc:2454] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7d70991ed2f4a0caf6107788245484a in term 0.
I20260806 22:43:56.801751 29468 raft_consensus.cc:2781] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:56.801810 29468 raft_consensus.cc:482] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:56.801834 29468 raft_consensus.cc:3037] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:56.802512 29468 raft_consensus.cc:504] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.802623 29468 leader_election.cc:288] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [CANDIDATE]: Term 1 election: Requested vote from peers 44eb26f579794a689c1e4e589e41f946 (127.0.186.194:45595), 3233312369db4200b49bd8d270d0f44d (127.0.186.195:45899)
I20260806 22:43:56.802789 29358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c312ef1744e4142b28d43b52dc968be" candidate_uuid: "f7d70991ed2f4a0caf6107788245484a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3233312369db4200b49bd8d270d0f44d"
I20260806 22:43:56.802807 29117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c312ef1744e4142b28d43b52dc968be" candidate_uuid: "f7d70991ed2f4a0caf6107788245484a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44eb26f579794a689c1e4e589e41f946"
I20260806 22:43:56.802860 29358 raft_consensus.cc:3037] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:56.802879 29117 raft_consensus.cc:3037] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:56.803539 29358 raft_consensus.cc:2454] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7d70991ed2f4a0caf6107788245484a in term 1.
I20260806 22:43:56.803638 29117 raft_consensus.cc:2454] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7d70991ed2f4a0caf6107788245484a in term 1.
I20260806 22:43:56.803720 28785 leader_election.cc:302] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3233312369db4200b49bd8d270d0f44d, f7d70991ed2f4a0caf6107788245484a; no voters:
I20260806 22:43:56.803805 29468 raft_consensus.cc:2781] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:56.803949 29468 raft_consensus.cc:686] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [term 1 LEADER]: Becoming Leader. State: Replica: f7d70991ed2f4a0caf6107788245484a, State: Running, Role: LEADER
I20260806 22:43:56.804037 29468 consensus_queue.cc:237] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } }
I20260806 22:43:56.804785 28654 catalog_manager.cc:5495] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a reported cstate change: term changed from 0 to 1, leader changed from <none> to f7d70991ed2f4a0caf6107788245484a (127.0.186.193). New cstate: current_term: 1 leader_uuid: "f7d70991ed2f4a0caf6107788245484a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d70991ed2f4a0caf6107788245484a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 38029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 } health_report { overall_health: UNKNOWN } } }
I20260806 22:43:56.812260 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.810167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.0.186.193:38029)
0806 22:43:56.810214 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:56.810215 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:56.810223 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:43:56.810250 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:56.810250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:56.810265 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:56.810319 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:56.810464 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.810899 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.811641 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:56.811829 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:56.811922 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:56.812022 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:56.812104 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:56.812170 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:56.812202 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:43:56.812211 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:43:56.813923 29358 raft_consensus.cc:1261] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 1 FOLLOWER]: Refusing update from remote peer f7d70991ed2f4a0caf6107788245484a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:56.813923 29117 raft_consensus.cc:1261] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 1 FOLLOWER]: Refusing update from remote peer f7d70991ed2f4a0caf6107788245484a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:56.814289 29476 consensus_queue.cc:1035] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3233312369db4200b49bd8d270d0f44d" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:56.814427 29468 consensus_queue.cc:1035] T 3c312ef1744e4142b28d43b52dc968be P f7d70991ed2f4a0caf6107788245484a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44eb26f579794a689c1e4e589e41f946" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:56.817472 29481 mvcc.cc:205] Tried to move back new op lower bound from 7315686345987371008 to 7315686345949876224. Current Snapshot: MvccSnapshot[applied={T|T < 7315686345987371008}]
I20260806 22:43:56.819013 28654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:56.819092 28654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:56.819729 28654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb4a583e44d84471ad9ae32e2603bd27: Sending DeleteTablet for 3 replicas of tablet 3c312ef1744e4142b28d43b52dc968be
I20260806 22:43:56.820009 28855 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c312ef1744e4142b28d43b52dc968be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:56 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:55080
I20260806 22:43:56.820015 29086 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c312ef1744e4142b28d43b52dc968be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:56 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:40728
I20260806 22:43:56.820240 29493 tablet_replica.cc:330] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: stopping tablet replica
I20260806 22:43:56.820338 29493 raft_consensus.cc:2229] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:56.820384 29316 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c312ef1744e4142b28d43b52dc968be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:56 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:56092
I20260806 22:43:56.820431 29493 raft_consensus.cc:2258] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:56.820554 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 28758
I20260806 22:43:56.821681 29493 ts_tablet_manager.cc:1905] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:56.821144 29496 tablet_replica.cc:330] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: stopping tablet replica
I20260806 22:43:56.823568 29496 raft_consensus.cc:2229] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:56.823673 29496 raft_consensus.cc:2258] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:56.823737 29493 ts_tablet_manager.cc:1918] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:56.825466 29493 log.cc:1196] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/3c312ef1744e4142b28d43b52dc968be
I20260806 22:43:56.825511 29496 ts_tablet_manager.cc:1905] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:56.825891 29493 ts_tablet_manager.cc:1939] T 3c312ef1744e4142b28d43b52dc968be P 44eb26f579794a689c1e4e589e41f946: Deleting consensus metadata
I20260806 22:43:56.827190 29496 ts_tablet_manager.cc:1918] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:56.827283 29496 log.cc:1196] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/3c312ef1744e4142b28d43b52dc968be
I20260806 22:43:56.827536 29496 ts_tablet_manager.cc:1939] T 3c312ef1744e4142b28d43b52dc968be P 3233312369db4200b49bd8d270d0f44d: Deleting consensus metadata
I20260806 22:43:56.828299 28627 catalog_manager.cc:4841] TS 3233312369db4200b49bd8d270d0f44d (127.0.186.195:45899): tablet 3c312ef1744e4142b28d43b52dc968be (table test-table [id=8577f70263fc4d43883e0e2cfbb13beb]) successfully deleted
I20260806 22:43:56.830346 28631 catalog_manager.cc:4841] TS 44eb26f579794a689c1e4e589e41f946 (127.0.186.194:45595): tablet 3c312ef1744e4142b28d43b52dc968be (table test-table [id=8577f70263fc4d43883e0e2cfbb13beb]) successfully deleted
W20260806 22:43:56.834257 28629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:38029 (error 108)
I20260806 22:43:56.834401 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 28975
I20260806 22:43:56.836148 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:56.834628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:38029 (local address 127.0.0.1:55102)
0806 22:43:56.836049 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:56.836106 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1396}
W20260806 22:43:56.836301 28629 catalog_manager.cc:4584] TS f7d70991ed2f4a0caf6107788245484a (127.0.186.193:38029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c312ef1744e4142b28d43b52dc968be: Network error: Client connection negotiation failed: client connection to 127.0.186.193:38029: connect: Connection refused (error 111)
I20260806 22:43:56.844347 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 29202
I20260806 22:43:56.857095 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 28597
2026-08-06T22:43:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1985 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:56Z Disabled control of system clock
I20260806 22:43:56.887691 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42913
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:42913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:56.968294 29504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:56.968506 29504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:56.968526 29504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:56.969642 29504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:56.969678 29504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:56.969689 29504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:56.969700 29504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:56.969712 29504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:56.970978 29504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:42913
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:42913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:56.971175 29504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:56.971467 29504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:56.975909 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.976059 29504 server_base.cc:1034] running on GCE node
W20260806 22:43:56.975936 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:56.975936 29519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:56.976390 29504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:56.976684 29504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:56.977810 29504 hybrid_clock.cc:648] HybridClock initialized: now 1786056236977794 us; error 33 us; skew 500 ppm
I20260806 22:43:56.978986 29504 webserver.cc:460] Webserver started at http://127.0.186.254:42717/ using document root <none> and password file <none>
I20260806 22:43:56.979204 29504 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:56.979250 29504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:56.979399 29504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:56.980418 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "1eec5789c32447bbb4d72d38b8c7942d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "793de4ea9db38869b3e4374f2c000b8d"
server_key_iv: "0ffe05404aa2c724cc3e4f2968b306d3"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.980762 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "1eec5789c32447bbb4d72d38b8c7942d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "793de4ea9db38869b3e4374f2c000b8d"
server_key_iv: "0ffe05404aa2c724cc3e4f2968b306d3"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.982290 29504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:56.983373 29531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:56.983692 29504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:56.983790 29504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "1eec5789c32447bbb4d72d38b8c7942d"
format_stamp: "Formatted at 2026-08-06 22:43:56 on dist-test-slave-gjrz"
server_key: "793de4ea9db38869b3e4374f2c000b8d"
server_key_iv: "0ffe05404aa2c724cc3e4f2968b306d3"
server_key_version: "encryptionkey@0"
I20260806 22:43:56.983866 29504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:57.020730 29504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:57.021027 29504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:57.021167 29504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:57.024920 29504 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:42913
I20260806 22:43:57.024960 29616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:42913 every 8 connection(s)
I20260806 22:43:57.025285 29504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:57.026257 29620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:57.028601 29620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: Bootstrap starting.
I20260806 22:43:57.029191 29620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:57.029440 29620 log.cc:824] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:57.030055 29620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: No bootstrap required, opened a new log
I20260806 22:43:57.031664 29620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } }
I20260806 22:43:57.031800 29620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:57.031821 29620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eec5789c32447bbb4d72d38b8c7942d, State: Initialized, Role: FOLLOWER
I20260806 22:43:57.031934 29620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } }
I20260806 22:43:57.032011 29620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:57.032038 29620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:57.032073 29620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:57.032235 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 29504
I20260806 22:43:57.032337 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:57.032545 747 external_mini_cluster.cc:1428] Setting key 5317cec0b799a24399ce1d65062a21a7
I20260806 22:43:57.032707 29620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } }
I20260806 22:43:57.032807 29620 leader_election.cc:302] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eec5789c32447bbb4d72d38b8c7942d; no voters:
I20260806 22:43:57.033020 29620 leader_election.cc:288] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:57.033094 29624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:57.033285 29624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [term 1 LEADER]: Becoming Leader. State: Replica: 1eec5789c32447bbb4d72d38b8c7942d, State: Running, Role: LEADER
I20260806 22:43:57.033334 29620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:57.033404 29624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } }
I20260806 22:43:57.033711 29624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eec5789c32447bbb4d72d38b8c7942d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } } }
I20260806 22:43:57.033851 29624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:57.034029 29624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eec5789c32447bbb4d72d38b8c7942d. Latest consensus state: current_term: 1 leader_uuid: "1eec5789c32447bbb4d72d38b8c7942d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec5789c32447bbb4d72d38b8c7942d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 42913 } } }
I20260806 22:43:57.034130 29624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d [sys.catalog]: This master's current role is: LEADER
W20260806 22:43:57.035193 29656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:43:57.035250 29656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:43:57.035538 29641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:57.035719 29641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:57.037092 29641 catalog_manager.cc:1347] Generated new cluster ID: 6afb4b69f3724d21b7c44f984eacb3d4
I20260806 22:43:57.037139 29641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:43:57.040699 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.033027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.0.186.254:42913)
0806 22:43:57.033260 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.033269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.033311 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:43:57.033512 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.033521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.033596 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.033726 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:57.034080 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.035401 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.036445 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.036958 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.039396 (+ 2438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.040059 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:57.040063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:57.040063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:57.040250 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:57.040289 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:57.040301 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:57.040362 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.040436 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.040509 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260806 22:43:57.058532 29641 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:57.059044 29641 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:57.062750 29641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: Generated new TSK 0
I20260806 22:43:57.062904 29641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:43:57.070679 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--builtin_ntp_servers=127.0.186.212:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:57.152310 29659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:57.152519 29659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:57.152539 29659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:57.153724 29659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:57.153766 29659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:57.153790 29659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:57.155293 29659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:57.155522 29659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:57.155807 29659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:57.160717 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:57.160710 29673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:57.160727 29674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:57.160867 29659 server_base.cc:1034] running on GCE node
I20260806 22:43:57.161206 29659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:57.161401 29659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:57.162549 29659 hybrid_clock.cc:648] HybridClock initialized: now 1786056237162532 us; error 28 us; skew 500 ppm
I20260806 22:43:57.163745 29659 webserver.cc:460] Webserver started at http://127.0.186.193:33091/ using document root <none> and password file <none>
I20260806 22:43:57.163930 29659 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:57.163969 29659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:57.164067 29659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:57.165057 29659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "5e98b619fea746da9c56e1ad40e613dd"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "3bee39edc33ae3aa59580bbc5b31cda8"
server_key_iv: "51db47b863842d9c324d8f5251dbe67d"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.165345 29659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "5e98b619fea746da9c56e1ad40e613dd"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "3bee39edc33ae3aa59580bbc5b31cda8"
server_key_iv: "51db47b863842d9c324d8f5251dbe67d"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.166486 29659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:57.167152 29692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.167332 29659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:57.167423 29659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "5e98b619fea746da9c56e1ad40e613dd"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "3bee39edc33ae3aa59580bbc5b31cda8"
server_key_iv: "51db47b863842d9c324d8f5251dbe67d"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.167481 29659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:57.178273 29659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:57.178520 29659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:57.178614 29659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:57.178787 29659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:57.179049 29659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:57.179078 29659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.179097 29659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:57.179113 29659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.184669 29659 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:34523
I20260806 22:43:57.184741 29885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:34523 every 8 connection(s)
I20260806 22:43:57.184998 29659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:57.186004 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 29659
I20260806 22:43:57.186103 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:57.186307 747 external_mini_cluster.cc:1428] Setting key 11c413c7e910c98073722196711be782
I20260806 22:43:57.187330 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--builtin_ntp_servers=127.0.186.212:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:43:57.190689 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.185647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:52033 (local address 127.0.186.254:42913)
0806 22:43:57.185726 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.185729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.185916 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:43:57.186566 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.186567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.186583 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.186635 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:57.187519 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.188022 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.189301 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.189598 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.189672 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.190447 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:57.190448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:57.190449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:57.190453 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:57.190471 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:57.190475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:57.190528 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.190580 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.190613 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:43:57.190834 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.185619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42913 (local address 127.0.186.193:52033)
0806 22:43:57.185786 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.185830 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.186469 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.186704 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.186708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.186736 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:57.187389 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.187398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.188120 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.188121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.189207 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.189211 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.189694 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.189695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.189810 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.190304 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:57.190304 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:57.190332 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:57.190355 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:57.190532 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:57.190534 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:57.190536 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.190580 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.190656 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260806 22:43:57.191298 29895 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42913
I20260806 22:43:57.191428 29895 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:57.191605 29895 heartbeater.cc:511] Master 127.0.186.254:42913 requested a full tablet report, sending...
I20260806 22:43:57.192159 29557 ts_manager.cc:194] Registered new tserver with Master: 5e98b619fea746da9c56e1ad40e613dd (127.0.186.193:34523)
I20260806 22:43:57.193037 29557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.193:52033
W20260806 22:43:57.269418 29899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:57.269606 29899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:57.269625 29899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:57.270716 29899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:57.270761 29899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:57.270794 29899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:57.272289 29899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:57.272504 29899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:57.272760 29899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:57.277525 29907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:57.277566 29910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:57.277536 29908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:57.277660 29899 server_base.cc:1034] running on GCE node
I20260806 22:43:57.277918 29899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:57.278136 29899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:57.279297 29899 hybrid_clock.cc:648] HybridClock initialized: now 1786056237279280 us; error 27 us; skew 500 ppm
I20260806 22:43:57.280484 29899 webserver.cc:460] Webserver started at http://127.0.186.194:39819/ using document root <none> and password file <none>
I20260806 22:43:57.280689 29899 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:57.280738 29899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:57.280864 29899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:57.281880 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "34b6bf00e6134be69a8a041081452a7e"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "10482d7492980dd8f371122a711fb578"
server_key_iv: "b38f891d70d2e01cb0e27265e8dbb88a"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.282248 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "34b6bf00e6134be69a8a041081452a7e"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "10482d7492980dd8f371122a711fb578"
server_key_iv: "b38f891d70d2e01cb0e27265e8dbb88a"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.283463 29899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:57.284106 29917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.284248 29899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:57.284332 29899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "34b6bf00e6134be69a8a041081452a7e"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "10482d7492980dd8f371122a711fb578"
server_key_iv: "b38f891d70d2e01cb0e27265e8dbb88a"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.284392 29899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:57.302038 29899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:57.302307 29899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:57.302434 29899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:57.302644 29899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:57.302955 29899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:57.302991 29899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.303018 29899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:57.303036 29899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.308490 29899 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:43161
I20260806 22:43:57.308538 30107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:43161 every 8 connection(s)
I20260806 22:43:57.308842 29899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:43:57.313097 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 29899
I20260806 22:43:57.313198 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:57.313431 747 external_mini_cluster.cc:1428] Setting key 3a62075eb8b227f2d95b38005b359f52
I20260806 22:43:57.313695 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.309519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:34331 (local address 127.0.186.254:42913)
0806 22:43:57.309597 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.309600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.309776 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:43:57.310255 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.310256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.310267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.310317 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:57.310941 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.311442 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.312380 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.312572 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.312623 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.313424 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:57.313425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:57.313426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:57.313430 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:57.313443 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:57.313447 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:57.313491 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.313586 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.313615 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:43:57.313913 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.309491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42913 (local address 127.0.186.194:34331)
0806 22:43:57.309666 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.309699 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.310174 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.310380 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.310383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.310392 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:57.310875 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.310879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.311510 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.311512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.312307 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.312309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.312633 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.312634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.312736 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.313260 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:57.313260 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:57.313299 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:57.313328 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:57.313503 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:57.313509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:57.313512 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.313582 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.313673 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260806 22:43:57.314373 30110 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42913
I20260806 22:43:57.314458 30110 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:57.314455 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--builtin_ntp_servers=127.0.186.212:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:43:57.314643 30110 heartbeater.cc:511] Master 127.0.186.254:42913 requested a full tablet report, sending...
I20260806 22:43:57.315009 29557 ts_manager.cc:194] Registered new tserver with Master: 34b6bf00e6134be69a8a041081452a7e (127.0.186.194:43161)
I20260806 22:43:57.315418 29557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.194:34331
W20260806 22:43:57.394712 30116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:57.394905 30116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:57.394933 30116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:57.396070 30116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:57.396111 30116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:57.396143 30116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:57.397469 30116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:57.397692 30116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:57.397958 30116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:57.402693 30129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:57.402709 30122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:57.402751 30116 server_base.cc:1034] running on GCE node
W20260806 22:43:57.402705 30125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:57.403083 30116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:57.403357 30116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:57.404501 30116 hybrid_clock.cc:648] HybridClock initialized: now 1786056237404487 us; error 29 us; skew 500 ppm
I20260806 22:43:57.405692 30116 webserver.cc:460] Webserver started at http://127.0.186.195:41859/ using document root <none> and password file <none>
I20260806 22:43:57.405907 30116 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:57.405957 30116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:57.406087 30116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:57.407084 30116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "e699f1f8966841dcab01448a87acc170"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "9c5b0d220c0b1ad6d904a46a0262882e"
server_key_iv: "79c79a0ae66e79e43382ffe4c5972815"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.407438 30116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "e699f1f8966841dcab01448a87acc170"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "9c5b0d220c0b1ad6d904a46a0262882e"
server_key_iv: "79c79a0ae66e79e43382ffe4c5972815"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.408627 30116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:43:57.409298 30137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.409488 30116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:57.409579 30116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "e699f1f8966841dcab01448a87acc170"
format_stamp: "Formatted at 2026-08-06 22:43:57 on dist-test-slave-gjrz"
server_key: "9c5b0d220c0b1ad6d904a46a0262882e"
server_key_iv: "79c79a0ae66e79e43382ffe4c5972815"
server_key_version: "encryptionkey@0"
I20260806 22:43:57.409646 30116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:57.439950 30116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:57.440235 30116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:57.440363 30116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:57.440580 30116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:57.440923 30116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:57.440958 30116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.440987 30116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:57.441004 30116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:57.446321 30116 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:38299
I20260806 22:43:57.446376 30311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:38299 every 8 connection(s)
I20260806 22:43:57.446681 30116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:57.449895 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 30116
I20260806 22:43:57.450004 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:57.450205 747 external_mini_cluster.cc:1428] Setting key b6712708262130fcf32e8e402848a204
I20260806 22:43:57.451912 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.447354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:37561 (local address 127.0.186.254:42913)
0806 22:43:57.447409 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.447411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.447653 (+ 242us) server_negotiation.cc:407] Connection header received
0806 22:43:57.448115 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.448115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.448127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.448176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:57.448805 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.449298 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.450526 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.450816 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.450876 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.451697 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:57.451698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:57.451699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:57.451703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:57.451716 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:57.451720 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:57.451760 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.451821 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.451847 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:43:57.452075 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.447348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:42913 (local address 127.0.186.195:37561)
0806 22:43:57.447530 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.447575 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.448039 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.448228 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.448232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.448240 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:57.448730 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.448734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.449369 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.449370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.450425 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.450429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.450895 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.450896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.451033 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.451576 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:43:57.451576 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:57.451605 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:43:57.451628 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:57.451767 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:57.451769 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:57.451772 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.451815 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.451892 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260806 22:43:57.452467 30314 heartbeater.cc:347] Connected to a master server at 127.0.186.254:42913
I20260806 22:43:57.452560 30314 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:57.452703 30314 heartbeater.cc:511] Master 127.0.186.254:42913 requested a full tablet report, sending...
I20260806 22:43:57.453016 29557 ts_manager.cc:194] Registered new tserver with Master: e699f1f8966841dcab01448a87acc170 (127.0.186.195:38299)
I20260806 22:43:57.453339 29557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.195:37561
I20260806 22:43:57.461813 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:57.465850 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.463113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.0.186.254:42913)
0806 22:43:57.463170 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.463173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.463257 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:43:57.463344 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.463345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.463357 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.463398 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:57.463550 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.464016 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.464724 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.464913 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.464965 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.465583 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:57.465584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:57.465585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:43:57.465589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:57.465604 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:43:57.465607 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:57.465675 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.465763 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.465772 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:43:57.467391 29557 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:57.468861 29557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:57.474769 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.471078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.0.186.195:38299)
0806 22:43:57.471239 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.471243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.471254 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:57.471314 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.471317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.471332 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.471414 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:57.471694 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.472769 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.473759 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.474388 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.474458 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.474543 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.474659 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.474693 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260806 22:43:57.474773 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.470728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:38299 (local address 127.0.0.1:43158)
0806 22:43:57.470846 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.470887 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.470951 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.471435 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.471438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.471441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:57.471513 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.471515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.472863 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.472864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.473699 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.473701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.474469 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.474470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.474554 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.474567 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.474646 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.474700 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260806 22:43:57.475000 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.470728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:34523 (local address 127.0.0.1:47504)
0806 22:43:57.470889 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.470898 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.470950 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.471797 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.471798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.471800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:57.471841 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.471842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.472715 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.472716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.473834 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.473836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.474704 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.474705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.474762 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.474766 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.474793 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.474944 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260806 22:43:57.475293 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.470931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43161 (local address 127.0.0.1:41558)
0806 22:43:57.471370 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.471380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.471426 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.471722 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.471723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.471725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:57.471772 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.471773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.473003 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.473004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.474181 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.474183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.475065 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.475066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.475127 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.475131 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.475161 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.475199 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":65,"threads_started":1}
I20260806 22:43:57.475477 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.470824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.0.186.193:34523)
0806 22:43:57.471541 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.471545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.471554 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:57.471591 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.471593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.471606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.471682 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:57.471952 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.472635 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.473922 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.474618 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.475212 (+ 594us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.475328 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.475356 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.475400 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":103,"threads_started":1}
I20260806 22:43:57.475513 30217 tablet_service.cc:1467] Processing CreateTablet for tablet 427c41088da24a8d9feb371a9ed2b562 (DEFAULT_TABLE table=test-table [id=29fd1f3d447e442980f0b14fd8ea6dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:57.475698 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.471342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.0.186.194:43161)
0806 22:43:57.471558 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.471562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.471571 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:57.471609 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.471611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.471625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.471840 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:57.471894 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.472929 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.474260 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.474969 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.475456 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.475560 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.475586 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.475625 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":88,"threads_started":1}
I20260806 22:43:57.475873 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c41088da24a8d9feb371a9ed2b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:57.476085 29772 tablet_service.cc:1467] Processing CreateTablet for tablet 427c41088da24a8d9feb371a9ed2b562 (DEFAULT_TABLE table=test-table [id=29fd1f3d447e442980f0b14fd8ea6dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:57.476413 29772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c41088da24a8d9feb371a9ed2b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:57.478331 30335 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Bootstrap starting.
I20260806 22:43:57.478729 30336 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Bootstrap starting.
I20260806 22:43:57.479038 29997 tablet_service.cc:1467] Processing CreateTablet for tablet 427c41088da24a8d9feb371a9ed2b562 (DEFAULT_TABLE table=test-table [id=29fd1f3d447e442980f0b14fd8ea6dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:57.479183 30335 tablet_bootstrap.cc:654] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:57.479375 29997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c41088da24a8d9feb371a9ed2b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:57.479385 30336 tablet_bootstrap.cc:654] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:57.479528 30335 log.cc:824] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:57.479665 30336 log.cc:824] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:57.480260 30335 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: No bootstrap required, opened a new log
I20260806 22:43:57.480306 30335 ts_tablet_manager.cc:1397] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:43:57.480403 30336 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: No bootstrap required, opened a new log
I20260806 22:43:57.480460 30336 ts_tablet_manager.cc:1397] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:43:57.482072 30335 raft_consensus.cc:348] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.482187 30335 raft_consensus.cc:374] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:57.482206 30335 raft_consensus.cc:729] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e699f1f8966841dcab01448a87acc170, State: Initialized, Role: FOLLOWER
I20260806 22:43:57.482295 30335 consensus_queue.cc:260] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.482535 30335 ts_tablet_manager.cc:1428] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:57.482674 30314 heartbeater.cc:503] Master 127.0.186.254:42913 was elected leader, sending a full tablet report...
I20260806 22:43:57.482754 30339 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Bootstrap starting.
I20260806 22:43:57.482936 30336 raft_consensus.cc:348] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.483211 30336 raft_consensus.cc:374] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:57.483290 30336 raft_consensus.cc:729] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e98b619fea746da9c56e1ad40e613dd, State: Initialized, Role: FOLLOWER
I20260806 22:43:57.483371 30339 tablet_bootstrap.cc:654] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:57.483426 30336 consensus_queue.cc:260] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.483671 30336 ts_tablet_manager.cc:1428] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:43:57.483697 30339 log.cc:824] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:57.483817 29895 heartbeater.cc:503] Master 127.0.186.254:42913 was elected leader, sending a full tablet report...
I20260806 22:43:57.484391 30339 tablet_bootstrap.cc:492] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: No bootstrap required, opened a new log
I20260806 22:43:57.484454 30339 ts_tablet_manager.cc:1397] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:43:57.486161 30339 raft_consensus.cc:348] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.486279 30339 raft_consensus.cc:374] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:57.486308 30339 raft_consensus.cc:729] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b6bf00e6134be69a8a041081452a7e, State: Initialized, Role: FOLLOWER
I20260806 22:43:57.486414 30339 consensus_queue.cc:260] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.486672 30339 ts_tablet_manager.cc:1428] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:57.486712 30110 heartbeater.cc:503] Master 127.0.186.254:42913 was elected leader, sending a full tablet report...
W20260806 22:43:57.559424 30111 tablet.cc:2369] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:57.685650 29896 tablet.cc:2369] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:43:57.697189 30315 tablet.cc:2369] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:57.736092 30344 raft_consensus.cc:482] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:57.736238 30344 raft_consensus.cc:504] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.736601 30344 leader_election.cc:288] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e699f1f8966841dcab01448a87acc170 (127.0.186.195:38299), 34b6bf00e6134be69a8a041081452a7e (127.0.186.194:43161)
I20260806 22:43:57.740629 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.736851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:53813 (local address 127.0.186.194:43161)
0806 22:43:57.736900 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.736902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.736989 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:43:57.737055 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.737055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.737066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.737130 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:57.737334 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.738081 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.739677 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.740246 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.740328 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.740336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.740517 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.740567 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:43:57.740629 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.736862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:55309 (local address 127.0.186.195:38299)
0806 22:43:57.736911 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.736913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.736990 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:43:57.737055 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.737056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.737065 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.737136 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:57.737335 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.738108 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.739677 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.740256 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.740328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.740336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.740517 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.740555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:43:57.740664 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.736788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:38299 (local address 127.0.186.193:55309)
0806 22:43:57.736916 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.736928 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.736996 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.737150 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.737151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.737153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:57.737261 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.737264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.738184 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.738184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.739570 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.739572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.740339 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.740340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.740438 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.740446 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.740506 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.740555 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":120,"thread_start_us":61,"threads_started":1}
I20260806 22:43:57.740666 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.736794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:43161 (local address 127.0.186.193:53813)
0806 22:43:57.736916 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.736928 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:57.736996 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:57.737150 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:57.737151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:57.737153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:57.737261 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.737264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.738178 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.738180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.739563 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:57.739567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.740340 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.740341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:57.740438 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.740447 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:43:57.740506 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:43:57.740555 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":74,"thread_start_us":60,"threads_started":1}
I20260806 22:43:57.741091 30024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c41088da24a8d9feb371a9ed2b562" candidate_uuid: "5e98b619fea746da9c56e1ad40e613dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b6bf00e6134be69a8a041081452a7e" is_pre_election: true
I20260806 22:43:57.741250 30024 raft_consensus.cc:2454] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e98b619fea746da9c56e1ad40e613dd in term 0.
I20260806 22:43:57.741259 30247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c41088da24a8d9feb371a9ed2b562" candidate_uuid: "5e98b619fea746da9c56e1ad40e613dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e699f1f8966841dcab01448a87acc170" is_pre_election: true
I20260806 22:43:57.741417 30247 raft_consensus.cc:2454] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e98b619fea746da9c56e1ad40e613dd in term 0.
I20260806 22:43:57.741503 29699 leader_election.cc:302] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b6bf00e6134be69a8a041081452a7e, 5e98b619fea746da9c56e1ad40e613dd; no voters:
I20260806 22:43:57.741631 30344 raft_consensus.cc:2781] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:57.741696 30344 raft_consensus.cc:482] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:57.741712 30344 raft_consensus.cc:3037] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:57.742434 30344 raft_consensus.cc:504] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.742556 30344 leader_election.cc:288] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [CANDIDATE]: Term 1 election: Requested vote from peers e699f1f8966841dcab01448a87acc170 (127.0.186.195:38299), 34b6bf00e6134be69a8a041081452a7e (127.0.186.194:43161)
I20260806 22:43:57.742743 30024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c41088da24a8d9feb371a9ed2b562" candidate_uuid: "5e98b619fea746da9c56e1ad40e613dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b6bf00e6134be69a8a041081452a7e"
I20260806 22:43:57.742743 30247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c41088da24a8d9feb371a9ed2b562" candidate_uuid: "5e98b619fea746da9c56e1ad40e613dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e699f1f8966841dcab01448a87acc170"
I20260806 22:43:57.742810 30024 raft_consensus.cc:3037] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:57.742817 30247 raft_consensus.cc:3037] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:57.743588 30024 raft_consensus.cc:2454] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e98b619fea746da9c56e1ad40e613dd in term 1.
I20260806 22:43:57.743621 30247 raft_consensus.cc:2454] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e98b619fea746da9c56e1ad40e613dd in term 1.
I20260806 22:43:57.743748 29699 leader_election.cc:302] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b6bf00e6134be69a8a041081452a7e, 5e98b619fea746da9c56e1ad40e613dd; no voters:
I20260806 22:43:57.743837 30344 raft_consensus.cc:2781] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:57.743988 30344 raft_consensus.cc:686] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [term 1 LEADER]: Becoming Leader. State: Replica: 5e98b619fea746da9c56e1ad40e613dd, State: Running, Role: LEADER
I20260806 22:43:57.744098 30344 consensus_queue.cc:237] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } }
I20260806 22:43:57.744875 29557 catalog_manager.cc:5495] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e98b619fea746da9c56e1ad40e613dd (127.0.186.193). New cstate: current_term: 1 leader_uuid: "5e98b619fea746da9c56e1ad40e613dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e98b619fea746da9c56e1ad40e613dd" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 34523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 } health_report { overall_health: UNKNOWN } } }
I20260806 22:43:57.755339 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.753225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.0.186.193:34523)
0806 22:43:57.753264 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:57.753266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:57.753276 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:43:57.753336 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:57.753336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:57.753353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:57.753406 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:57.753500 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.753933 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.754714 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:57.754894 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:57.754944 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:57.755070 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:57.755151 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:57.755216 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:57.755248 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:43:57.755256 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:43:57.757012 30024 raft_consensus.cc:1261] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 1 FOLLOWER]: Refusing update from remote peer 5e98b619fea746da9c56e1ad40e613dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:57.757026 30247 raft_consensus.cc:1261] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 1 FOLLOWER]: Refusing update from remote peer 5e98b619fea746da9c56e1ad40e613dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:57.757313 30351 consensus_queue.cc:1035] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b6bf00e6134be69a8a041081452a7e" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:57.757432 30344 consensus_queue.cc:1035] T 427c41088da24a8d9feb371a9ed2b562 P 5e98b619fea746da9c56e1ad40e613dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e699f1f8966841dcab01448a87acc170" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 38299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:57.760164 30359 mvcc.cc:205] Tried to move back new op lower bound from 7315686349850542080 to 7315686349800382464. Current Snapshot: MvccSnapshot[applied={T|T < 7315686349850542080 or (T in {7315686349850542080})}]
I20260806 22:43:57.762390 29557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:57.762482 29557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:57.763134 29557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1eec5789c32447bbb4d72d38b8c7942d: Sending DeleteTablet for 3 replicas of tablet 427c41088da24a8d9feb371a9ed2b562
I20260806 22:43:57.763520 29772 tablet_service.cc:1514] Processing DeleteTablet for tablet 427c41088da24a8d9feb371a9ed2b562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:57 UTC) from {username='slave'} at 127.0.0.1:47504
I20260806 22:43:57.763517 29997 tablet_service.cc:1514] Processing DeleteTablet for tablet 427c41088da24a8d9feb371a9ed2b562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:57 UTC) from {username='slave'} at 127.0.0.1:41558
I20260806 22:43:57.763761 30370 tablet_replica.cc:330] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: stopping tablet replica
I20260806 22:43:57.763803 30217 tablet_service.cc:1514] Processing DeleteTablet for tablet 427c41088da24a8d9feb371a9ed2b562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:57 UTC) from {username='slave'} at 127.0.0.1:43158
I20260806 22:43:57.763947 30370 raft_consensus.cc:2229] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:57.764047 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 29659
I20260806 22:43:57.764127 30370 raft_consensus.cc:2258] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:57.764287 30371 tablet_replica.cc:330] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: stopping tablet replica
I20260806 22:43:57.764387 30371 raft_consensus.cc:2229] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:57.768187 30370 ts_tablet_manager.cc:1905] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:57.771188 30370 ts_tablet_manager.cc:1918] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:57.771294 30370 log.cc:1196] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/427c41088da24a8d9feb371a9ed2b562
I20260806 22:43:57.771199 30371 raft_consensus.cc:2258] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:57.771507 30370 ts_tablet_manager.cc:1939] T 427c41088da24a8d9feb371a9ed2b562 P 34b6bf00e6134be69a8a041081452a7e: Deleting consensus metadata
I20260806 22:43:57.771870 29537 catalog_manager.cc:4841] TS 34b6bf00e6134be69a8a041081452a7e (127.0.186.194:43161): tablet 427c41088da24a8d9feb371a9ed2b562 (table test-table [id=29fd1f3d447e442980f0b14fd8ea6dcb]) successfully deleted
I20260806 22:43:57.774600 30371 ts_tablet_manager.cc:1905] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:57.776116 30371 ts_tablet_manager.cc:1918] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:57.776186 30371 log.cc:1196] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/427c41088da24a8d9feb371a9ed2b562
I20260806 22:43:57.776377 30371 ts_tablet_manager.cc:1939] T 427c41088da24a8d9feb371a9ed2b562 P e699f1f8966841dcab01448a87acc170: Deleting consensus metadata
I20260806 22:43:57.776759 29541 catalog_manager.cc:4841] TS e699f1f8966841dcab01448a87acc170 (127.0.186.195:38299): tablet 427c41088da24a8d9feb371a9ed2b562 (table test-table [id=29fd1f3d447e442980f0b14fd8ea6dcb]) successfully deleted
W20260806 22:43:57.782588 29537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:34523 (error 108)
I20260806 22:43:57.783061 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:57.782942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:34523 (local address 127.0.0.1:47524)
0806 22:43:57.782991 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:57.783029 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:43:57.783141 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 29899
W20260806 22:43:57.783195 29537 catalog_manager.cc:4584] TS 5e98b619fea746da9c56e1ad40e613dd (127.0.186.193:34523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 427c41088da24a8d9feb371a9ed2b562: Network error: Client connection negotiation failed: client connection to 127.0.186.193:34523: connect: Connection refused (error 111)
I20260806 22:43:57.800068 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 30116
I20260806 22:43:57.810608 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 29504
2026-08-06T22:43:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (952 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:43:57 dist-test-slave-gjrz krb5kdc[30377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:43:57 dist-test-slave-gjrz krb5kdc[30377](info): set up 2 sockets
Aug 06 22:43:57 dist-test-slave-gjrz krb5kdc[30377](info): commencing operation
krb5kdc: starting...
W20260806 22:43:58.952665 747 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:43:59.064215 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35711
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:35711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:59.144912 30399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:59.145128 30399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:59.145161 30399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:59.146338 30399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:59.146382 30399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:59.146401 30399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:59.146417 30399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:59.146433 30399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:59.147951 30399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:35711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:59.148211 30399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:59.148407 30399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:59.152915 30412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.152911 30413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.152998 30417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.153113 30399 server_base.cc:1034] running on GCE node
I20260806 22:43:59.153337 30399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:59.153602 30399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:59.154737 30399 hybrid_clock.cc:648] HybridClock initialized: now 1786056239154725 us; error 30 us; skew 500 ppm
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:59.157141 30399 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:43:59.157567 30399 webserver.cc:460] Webserver started at http://127.0.186.254:38895/ using document root <none> and password file <none>
I20260806 22:43:59.157769 30399 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:59.157816 30399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:59.157943 30399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:59.158967 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "22e7abadb8ee418e982ce90f0c4a5037"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ea067db1f1949aaa0c31aa6ea7b78588"
server_key_iv: "7736fa2e08e73637e4c3538e5516295b"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.159360 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "22e7abadb8ee418e982ce90f0c4a5037"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ea067db1f1949aaa0c31aa6ea7b78588"
server_key_iv: "7736fa2e08e73637e4c3538e5516295b"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.160602 30399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:43:59.161307 30428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.161476 30399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:59.161559 30399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "22e7abadb8ee418e982ce90f0c4a5037"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ea067db1f1949aaa0c31aa6ea7b78588"
server_key_iv: "7736fa2e08e73637e4c3538e5516295b"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.161621 30399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:59.184983 30399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:59.187070 30399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:59.187211 30399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:59.191066 30399 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:35711
I20260806 22:43:59.191102 30505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:35711 every 8 connection(s)
I20260806 22:43:59.191464 30399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:59.192293 30507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:59.194545 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: Bootstrap starting.
I20260806 22:43:59.195111 30507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:59.195410 30507 log.cc:824] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:59.196064 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: No bootstrap required, opened a new log
I20260806 22:43:59.197642 30507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } }
I20260806 22:43:59.197777 30507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:59.197809 30507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e7abadb8ee418e982ce90f0c4a5037, State: Initialized, Role: FOLLOWER
I20260806 22:43:59.197952 30507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } }
I20260806 22:43:59.198036 30507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:43:59.198060 30507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:43:59.198108 30507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:59.198619 30507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } }
I20260806 22:43:59.198714 30507 leader_election.cc:302] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22e7abadb8ee418e982ce90f0c4a5037; no voters:
I20260806 22:43:59.198881 30507 leader_election.cc:288] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:43:59.198911 30513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:59.199055 30513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [term 1 LEADER]: Becoming Leader. State: Replica: 22e7abadb8ee418e982ce90f0c4a5037, State: Running, Role: LEADER
I20260806 22:43:59.199116 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 30399
I20260806 22:43:59.199128 30507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:43:59.199150 30513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } }
I20260806 22:43:59.199231 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:59.199468 747 external_mini_cluster.cc:1428] Setting key c02c579bdbbeb080261b80448d9dafa2
I20260806 22:43:59.199522 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22e7abadb8ee418e982ce90f0c4a5037. Latest consensus state: current_term: 1 leader_uuid: "22e7abadb8ee418e982ce90f0c4a5037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } } }
I20260806 22:43:59.199610 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [sys.catalog]: This master's current role is: LEADER
I20260806 22:43:59.199903 30515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22e7abadb8ee418e982ce90f0c4a5037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e7abadb8ee418e982ce90f0c4a5037" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35711 } } }
I20260806 22:43:59.199980 30515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037 [sys.catalog]: This master's current role is: LEADER
W20260806 22:43:59.200949 30544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:43:59.201005 30544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:43:59.201097 30526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:43:59.201270 30526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:43:59.203509 30526 catalog_manager.cc:1347] Generated new cluster ID: 6cfd4702bfaa4903b9ef08299ef0d84f
I20260806 22:43:59.203568 30526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:59.210348 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.200112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.0.186.254:35711)
0806 22:43:59.200294 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.200305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.200566 (+ 261us) server_negotiation.cc:407] Connection header received
0806 22:43:59.200681 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.200689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.200748 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.200885 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:59.201631 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.202678 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.203457 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.204043 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.204150 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.206843 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:59.206848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:59.206848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:59.206860 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:59.208776 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.209138 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.209141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.209141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.209271 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.209445 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.209446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.209446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.209760 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:59.209916 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.210003 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.210139 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 22:43:59.218701 30526 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:43:59.219340 30526 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:43:59.225359 30526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: Generated new TSK 0
I20260806 22:43:59.225529 30526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
Entry for principal kudu/127.0.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab.
I20260806 22:43:59.255599 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:59.337309 30553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:59.337510 30553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:59.337532 30553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:59.338653 30553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:59.338698 30553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:59.338723 30553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:43:59.340258 30553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.193.keytab
--principal=kudu/127.0.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:59.340478 30553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:59.340693 30553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:59.345616 30564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.345685 30553 server_base.cc:1034] running on GCE node
W20260806 22:43:59.345644 30560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.345860 30559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.346073 30553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:59.346274 30553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:59.347402 30553 hybrid_clock.cc:648] HybridClock initialized: now 1786056239347389 us; error 29 us; skew 500 ppm
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:59.349810 30553 init.cc:377] Logged in from keytab as kudu/127.0.186.193@KRBTEST.COM (short username kudu)
I20260806 22:43:59.350234 30553 webserver.cc:460] Webserver started at http://127.0.186.193:35039/ using document root <none> and password file <none>
I20260806 22:43:59.350435 30553 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:59.350481 30553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:59.350606 30553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:59.351637 30553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "ef1df8ab856242ef9c9172bb8d8048cc"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ab167edba1a1ec4e681893d471675aaa"
server_key_iv: "cc1fb616a103976c146e62410cef4157"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.351959 30553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "ef1df8ab856242ef9c9172bb8d8048cc"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ab167edba1a1ec4e681893d471675aaa"
server_key_iv: "cc1fb616a103976c146e62410cef4157"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.353152 30553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:43:59.353765 30582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.353924 30553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:43:59.354001 30553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "ef1df8ab856242ef9c9172bb8d8048cc"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "ab167edba1a1ec4e681893d471675aaa"
server_key_iv: "cc1fb616a103976c146e62410cef4157"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.354069 30553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:59.379470 30553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:59.381443 30553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:59.381585 30553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:59.381788 30553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:59.382090 30553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:59.382133 30553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.382160 30553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:59.382179 30553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.387540 30553 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:44465
I20260806 22:43:59.387610 30761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:44465 every 8 connection(s)
I20260806 22:43:59.387871 30553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:43:59.390827 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 30553
I20260806 22:43:59.390945 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:43:59.391131 747 external_mini_cluster.cc:1428] Setting key 813c54f18b8bc6644232b9fe5b4d7080
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.193@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:59.396688 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.388507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:40653 (local address 127.0.186.254:35711)
0806 22:43:59.388580 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.388582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.388744 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:43:59.389603 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.389604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.389618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.389666 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:59.390198 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.390713 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.391854 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.392141 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.392211 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.394196 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:59.394198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:59.394198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:59.394205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:59.395375 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.395732 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.395733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.395733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.395793 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.396098 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.396099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.396099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.396204 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:59.396251 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.396555 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.396627 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:43:59.396916 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.388468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35711 (local address 127.0.186.193:40653)
0806 22:43:59.388632 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.388668 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.389513 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.389719 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.389722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.389948 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:59.390130 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.390134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.390777 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.390778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.391777 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.391780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.392213 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.392214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.392307 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.392733 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:59.392741 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:59.394111 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:59.395451 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.395458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.395460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.395667 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.395872 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.395873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.395873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.396026 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.396260 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:59.396266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:59.396425 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.396535 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.396703 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260806 22:43:59.397401 30762 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35711
I20260806 22:43:59.397490 30762 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:59.397650 30762 heartbeater.cc:511] Master 127.0.186.254:35711 requested a full tablet report, sending...
I20260806 22:43:59.398144 30455 ts_manager.cc:194] Registered new tserver with Master: ef1df8ab856242ef9c9172bb8d8048cc (127.0.186.193:44465)
I20260806 22:43:59.398739 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.193@KRBTEST.COM'} at 127.0.186.193:40653
WARNING: no policy specified for kudu/127.0.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
Entry for principal kudu/127.0.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab.
I20260806 22:43:59.414660 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:59.495975 30768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:59.496232 30768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:59.496268 30768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:59.497818 30768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:59.497864 30768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:59.497890 30768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:43:59.499639 30768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.194.keytab
--principal=kudu/127.0.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:59.499861 30768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:59.500061 30768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:59.504757 30775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.504884 30768 server_base.cc:1034] running on GCE node
W20260806 22:43:59.504757 30778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.504786 30774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.505180 30768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:59.505446 30768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:59.506582 30768 hybrid_clock.cc:648] HybridClock initialized: now 1786056239506570 us; error 27 us; skew 500 ppm
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:59.511545 30768 init.cc:377] Logged in from keytab as kudu/127.0.186.194@KRBTEST.COM (short username kudu)
I20260806 22:43:59.511945 30768 webserver.cc:460] Webserver started at http://127.0.186.194:43261/ using document root <none> and password file <none>
I20260806 22:43:59.512171 30768 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:59.512215 30768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:59.512305 30768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:59.513244 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "53ab064dd5ad4a12bb81957812ac806a"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "199409145d48c57fe3a5d77ee157bcaf"
server_key_iv: "bea24a46f9f64c4d4f0f9b150866fdf6"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.513538 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "53ab064dd5ad4a12bb81957812ac806a"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "199409145d48c57fe3a5d77ee157bcaf"
server_key_iv: "bea24a46f9f64c4d4f0f9b150866fdf6"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.514680 30768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:59.515308 30788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.515455 30768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.515525 30768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "53ab064dd5ad4a12bb81957812ac806a"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "199409145d48c57fe3a5d77ee157bcaf"
server_key_iv: "bea24a46f9f64c4d4f0f9b150866fdf6"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.515606 30768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:59.532730 30768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:59.534793 30768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:59.534916 30768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:59.535130 30768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:59.535451 30768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:59.535487 30768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.535516 30768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:59.535542 30768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.541036 30768 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:37987
I20260806 22:43:59.541105 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:37987 every 8 connection(s)
I20260806 22:43:59.541378 30768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.194@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:59.549310 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 30768
I20260806 22:43:59.549424 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:43:59.549641 747 external_mini_cluster.cc:1428] Setting key 33be233e7762ef55c98ffd54cb7d9685
I20260806 22:43:59.550616 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.542094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45225 (local address 127.0.186.254:35711)
0806 22:43:59.542152 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.542154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.542347 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:43:59.543296 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.543298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.543311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.543363 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:59.543976 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.544472 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.545477 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.545674 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.545731 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.547968 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:59.547970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:59.547970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:59.547978 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:59.549120 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.549591 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.549592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.549592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.549645 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.549965 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.549966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.549967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.550118 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:59.550182 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.550443 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.550530 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:43:59.550755 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.542040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35711 (local address 127.0.186.194:45225)
0806 22:43:59.542220 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.542256 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.543163 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.543433 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.543437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.543689 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:59.543879 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.543883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.544554 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.544555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.545393 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.545396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.545744 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.545745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.545869 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.546278 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:59.546292 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:59.547860 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:59.549197 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.549205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.549208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.549520 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.549764 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.549765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.549765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.549885 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.550175 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:59.550181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:59.550329 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.550413 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.550573 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260806 22:43:59.551160 30964 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35711
I20260806 22:43:59.551246 30964 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:59.551440 30964 heartbeater.cc:511] Master 127.0.186.254:35711 requested a full tablet report, sending...
I20260806 22:43:59.551805 30455 ts_manager.cc:194] Registered new tserver with Master: 53ab064dd5ad4a12bb81957812ac806a (127.0.186.194:37987)
I20260806 22:43:59.552285 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.194@KRBTEST.COM'} at 127.0.186.194:45225
WARNING: no policy specified for kudu/127.0.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
Entry for principal kudu/127.0.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab.
I20260806 22:43:59.575836 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:43:59.656159 30975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:59.656350 30975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:59.656373 30975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:59.657572 30975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:59.657615 30975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:59.657642 30975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:43:59.659153 30975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/krb5kdc/kudu_127.0.186.195.keytab
--principal=kudu/127.0.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:59.659543 30975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:59.659762 30975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:59.664456 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.664417 30983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.664430 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.664746 30975 server_base.cc:1034] running on GCE node
I20260806 22:43:59.664953 30975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:59.665174 30975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:59.666303 30975 hybrid_clock.cc:648] HybridClock initialized: now 1786056239666287 us; error 31 us; skew 500 ppm
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:43:59.668645 30975 init.cc:377] Logged in from keytab as kudu/127.0.186.195@KRBTEST.COM (short username kudu)
I20260806 22:43:59.669067 30975 webserver.cc:460] Webserver started at http://127.0.186.195:42963/ using document root <none> and password file <none>
I20260806 22:43:59.669251 30975 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:59.669292 30975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:59.669404 30975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:59.670589 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "c337c3c189bc4c678cd15fec8c8c7d53"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6451dba0cd806f73c9da59288eff8a21"
server_key_iv: "dda32e45cf99d0bbb3e6746230059f87"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.670953 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "c337c3c189bc4c678cd15fec8c8c7d53"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6451dba0cd806f73c9da59288eff8a21"
server_key_iv: "dda32e45cf99d0bbb3e6746230059f87"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.672326 30975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:43:59.672950 30997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.673086 30975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:59.673163 30975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "c337c3c189bc4c678cd15fec8c8c7d53"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6451dba0cd806f73c9da59288eff8a21"
server_key_iv: "dda32e45cf99d0bbb3e6746230059f87"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.673229 30975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:59.701155 30975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:59.703179 30975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:59.703327 30975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:59.703531 30975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:43:59.703804 30975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:43:59.703836 30975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.703856 30975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:43:59.703879 30975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.709301 30975 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:33909
I20260806 22:43:59.709357 31166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:33909 every 8 connection(s)
I20260806 22:43:59.709633 30975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:43:59.710518 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 30975
I20260806 22:43:59.710629 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:43:59.710800 747 external_mini_cluster.cc:1428] Setting key 4e7bf18ae7aa4559e3f07302a4d5a00b
Aug 06 22:43:59 dist-test-slave-gjrz krb5kdc[30377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.195@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:43:59.718150 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.710283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:48575 (local address 127.0.186.254:35711)
0806 22:43:59.710344 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.710346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.710562 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:43:59.711644 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.711645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.711656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.711707 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:59.712310 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.712807 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.713754 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.713946 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.713999 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.715885 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:59.715886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:59.715886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:59.715894 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:59.716896 (+ 1002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.717256 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.717257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.717258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.717314 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.717522 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.717523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.717524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.717640 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:59.717698 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.717996 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.718090 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:43:59.718298 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.710249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35711 (local address 127.0.186.195:48575)
0806 22:43:59.710419 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.710467 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.711548 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.711774 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.711777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.712015 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:43:59.712231 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.712235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.712886 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.712887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.713675 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.713678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.714013 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.714014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.714132 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.714513 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:43:59.714521 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:43:59.715800 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:43:59.716968 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.716975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.716977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.717190 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.717369 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:43:59.717370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:43:59.717370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:43:59.717466 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:43:59.717696 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:43:59.717704 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:43:59.717829 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.717943 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.718111 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260806 22:43:59.718698 31167 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35711
I20260806 22:43:59.718781 31167 heartbeater.cc:464] Registering TS with master...
I20260806 22:43:59.718943 31167 heartbeater.cc:511] Master 127.0.186.254:35711 requested a full tablet report, sending...
I20260806 22:43:59.719324 30455 ts_manager.cc:194] Registered new tserver with Master: c337c3c189bc4c678cd15fec8c8c7d53 (127.0.186.195:33909)
I20260806 22:43:59.719681 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.186.195@KRBTEST.COM'} at 127.0.186.195:48575
I20260806 22:43:59.722246 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:43:59.728626 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.723478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.0.186.254:35711)
0806 22:43:59.723525 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.723526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.723599 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:43:59.723653 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.723654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.723662 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.723707 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:43:59.724158 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.724605 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.725315 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.725594 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.725643 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.726645 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:43:59.726646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:43:59.726646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:43:59.726651 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:43:59.727700 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.728004 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.728005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.728005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.728067 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:43:59.728210 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:43:59.728210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:43:59.728210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:43:59.728339 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:43:59.728423 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.728508 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.728569 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:43:59.730082 30455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:43:59.731669 30455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:43:59.737864 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.733772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:33909 (local address 127.0.0.1:39816)
0806 22:43:59.734337 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.734345 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.734387 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.734626 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.734627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.734629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:59.734730 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.734732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.735620 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.735621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.736787 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.736790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.737576 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.737657 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.737671 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.737741 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.737791 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":83,"threads_started":1}
I20260806 22:43:59.737875 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.733991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.0.186.195:33909)
0806 22:43:59.734146 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.734150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.734400 (+ 250us) server_negotiation.cc:407] Connection header received
0806 22:43:59.734459 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.734462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.734475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.734537 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:59.734804 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.735524 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.736863 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.737488 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737562 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.737619 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.737762 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.737803 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260806 22:43:59.738202 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.733760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44465 (local address 127.0.0.1:38998)
0806 22:43:59.734067 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.734105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.734158 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.734597 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.734601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.734602 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:59.734676 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.734677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.736004 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.736005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.737004 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.737006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.737978 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.738042 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.738045 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.738100 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.738138 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":8,"thread_start_us":63,"threads_started":1}
I20260806 22:43:59.738251 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.734175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.0.186.193:44465)
0806 22:43:59.734345 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.734350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.734361 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:43:59.734400 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.734402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.734418 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.734487 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:59.734754 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.735894 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737102 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.737821 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737883 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.737938 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.738132 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.738173 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260806 22:43:59.738996 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.733760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:37987 (local address 127.0.0.1:37782)
0806 22:43:59.734228 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.734235 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.734276 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.734592 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.734598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.734601 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:59.734672 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.734674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.737094 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.737840 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.737843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.738789 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.738790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.738850 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.738853 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.738895 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.738934 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260806 22:43:59.738967 31076 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b12234fc14e9e956bd225b079f40f (DEFAULT_TABLE table=test-table [id=104bb6e7b438401ca531251bbaf68e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:59.738972 30654 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b12234fc14e9e956bd225b079f40f (DEFAULT_TABLE table=test-table [id=104bb6e7b438401ca531251bbaf68e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:59.739024 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.733877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37782 (local address 127.0.186.194:37987)
0806 22:43:59.734040 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.734045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.734299 (+ 254us) server_negotiation.cc:407] Connection header received
0806 22:43:59.734350 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.734353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.734385 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.734469 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:59.734758 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.735898 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.737932 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.738708 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.738773 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.738825 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.738919 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.738955 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260806 22:43:59.739337 31076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b12234fc14e9e956bd225b079f40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:59.739337 30654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b12234fc14e9e956bd225b079f40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:59.739735 30862 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b12234fc14e9e956bd225b079f40f (DEFAULT_TABLE table=test-table [id=104bb6e7b438401ca531251bbaf68e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:43:59.740015 30862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b12234fc14e9e956bd225b079f40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:43:59.742477 31190 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Bootstrap starting.
I20260806 22:43:59.742764 31193 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: Bootstrap starting.
I20260806 22:43:59.743172 31190 tablet_bootstrap.cc:654] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:59.743335 31193 tablet_bootstrap.cc:654] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:59.743427 31194 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Bootstrap starting.
I20260806 22:43:59.743786 31190 log.cc:824] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:59.743831 31193 log.cc:824] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:59.744076 31194 tablet_bootstrap.cc:654] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Neither blocks nor log segments found. Creating new log.
I20260806 22:43:59.744590 31190 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: No bootstrap required, opened a new log
I20260806 22:43:59.744620 31194 log.cc:824] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Log is configured to *not* fsync() on all Append() calls
I20260806 22:43:59.744659 31193 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: No bootstrap required, opened a new log
I20260806 22:43:59.744666 31190 ts_tablet_manager.cc:1397] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:43:59.744721 31193 ts_tablet_manager.cc:1397] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:43:59.745428 31194 tablet_bootstrap.cc:492] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: No bootstrap required, opened a new log
I20260806 22:43:59.745492 31194 ts_tablet_manager.cc:1397] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:43:59.746326 31193 raft_consensus.cc:348] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.746470 31193 raft_consensus.cc:374] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:59.746502 31193 raft_consensus.cc:729] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1df8ab856242ef9c9172bb8d8048cc, State: Initialized, Role: FOLLOWER
I20260806 22:43:59.746644 31193 consensus_queue.cc:260] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.746551 31190 raft_consensus.cc:348] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.746719 31190 raft_consensus.cc:374] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:59.746762 31190 raft_consensus.cc:729] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ab064dd5ad4a12bb81957812ac806a, State: Initialized, Role: FOLLOWER
I20260806 22:43:59.746897 31193 ts_tablet_manager.cc:1428] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:43:59.746889 31190 consensus_queue.cc:260] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.747005 30762 heartbeater.cc:503] Master 127.0.186.254:35711 was elected leader, sending a full tablet report...
I20260806 22:43:59.747159 31190 ts_tablet_manager.cc:1428] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:43:59.747300 30964 heartbeater.cc:503] Master 127.0.186.254:35711 was elected leader, sending a full tablet report...
I20260806 22:43:59.747385 31194 raft_consensus.cc:348] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.747556 31194 raft_consensus.cc:374] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:43:59.747596 31194 raft_consensus.cc:729] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c337c3c189bc4c678cd15fec8c8c7d53, State: Initialized, Role: FOLLOWER
I20260806 22:43:59.747689 31194 consensus_queue.cc:260] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.747942 31194 ts_tablet_manager.cc:1428] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:43:59.748003 31167 heartbeater.cc:503] Master 127.0.186.254:35711 was elected leader, sending a full tablet report...
W20260806 22:43:59.791908 30965 tablet.cc:2369] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:43:59.798203 31201 raft_consensus.cc:482] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:43:59.798312 31201 raft_consensus.cc:504] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.798635 31201 leader_election.cc:288] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef1df8ab856242ef9c9172bb8d8048cc (127.0.186.193:44465), c337c3c189bc4c678cd15fec8c8c7d53 (127.0.186.195:33909)
I20260806 22:43:59.801563 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.798892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:57359 (local address 127.0.186.193:44465)
0806 22:43:59.799000 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.799003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.799013 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:43:59.799042 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.799043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.799054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.799110 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:59.799301 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.799851 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.800842 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.801223 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.801286 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.801295 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.801442 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.801490 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:43:59.801568 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.798780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44465 (local address 127.0.186.194:57359)
0806 22:43:59.798836 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.798848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.798904 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.799123 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.799124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.799126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:59.799200 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.799202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.799923 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.799924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.800765 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.800768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.801298 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.801300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.801381 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.801385 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.801429 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.801486 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":14}
I20260806 22:43:59.801573 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.798905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:45561 (local address 127.0.186.195:33909)
0806 22:43:59.798969 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.798971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.799068 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:43:59.799125 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.799126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.799135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.799193 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:43:59.799360 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.799867 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.800890 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.801260 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.801321 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.801329 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.801471 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.801509 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:43:59.801579 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.798805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:33909 (local address 127.0.186.194:45561)
0806 22:43:59.799002 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.799011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:43:59.799072 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:43:59.799200 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:43:59.799201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:43:59.799202 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:43:59.799292 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.799293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.799933 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.799934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.800816 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:43:59.800819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.801332 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.801333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:43:59.801405 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.801408 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:43:59.801453 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:43:59.801492 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
I20260806 22:43:59.801923 31102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b12234fc14e9e956bd225b079f40f" candidate_uuid: "53ab064dd5ad4a12bb81957812ac806a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" is_pre_election: true
I20260806 22:43:59.802042 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b12234fc14e9e956bd225b079f40f" candidate_uuid: "53ab064dd5ad4a12bb81957812ac806a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" is_pre_election: true
I20260806 22:43:59.802084 31102 raft_consensus.cc:2454] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab064dd5ad4a12bb81957812ac806a in term 0.
I20260806 22:43:59.802186 30686 raft_consensus.cc:2454] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab064dd5ad4a12bb81957812ac806a in term 0.
I20260806 22:43:59.802309 30794 leader_election.cc:302] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab064dd5ad4a12bb81957812ac806a, c337c3c189bc4c678cd15fec8c8c7d53; no voters:
I20260806 22:43:59.802448 31201 raft_consensus.cc:2781] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:43:59.802510 31201 raft_consensus.cc:482] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:43:59.802538 31201 raft_consensus.cc:3037] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:59.803205 31201 raft_consensus.cc:504] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.803409 31201 leader_election.cc:288] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [CANDIDATE]: Term 1 election: Requested vote from peers ef1df8ab856242ef9c9172bb8d8048cc (127.0.186.193:44465), c337c3c189bc4c678cd15fec8c8c7d53 (127.0.186.195:33909)
I20260806 22:43:59.803483 31102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b12234fc14e9e956bd225b079f40f" candidate_uuid: "53ab064dd5ad4a12bb81957812ac806a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c337c3c189bc4c678cd15fec8c8c7d53"
I20260806 22:43:59.803483 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b12234fc14e9e956bd225b079f40f" candidate_uuid: "53ab064dd5ad4a12bb81957812ac806a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1df8ab856242ef9c9172bb8d8048cc"
I20260806 22:43:59.803555 31102 raft_consensus.cc:3037] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:59.803568 30686 raft_consensus.cc:3037] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:43:59.804415 31102 raft_consensus.cc:2454] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab064dd5ad4a12bb81957812ac806a in term 1.
I20260806 22:43:59.804415 30686 raft_consensus.cc:2454] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab064dd5ad4a12bb81957812ac806a in term 1.
I20260806 22:43:59.804597 30794 leader_election.cc:302] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab064dd5ad4a12bb81957812ac806a, c337c3c189bc4c678cd15fec8c8c7d53; no voters:
I20260806 22:43:59.804713 31201 raft_consensus.cc:2781] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:43:59.804888 31201 raft_consensus.cc:686] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 1 LEADER]: Becoming Leader. State: Replica: 53ab064dd5ad4a12bb81957812ac806a, State: Running, Role: LEADER
I20260806 22:43:59.804982 31201 consensus_queue.cc:237] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } }
I20260806 22:43:59.805773 30454 catalog_manager.cc:5495] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ab064dd5ad4a12bb81957812ac806a (127.0.186.194). New cstate: current_term: 1 leader_uuid: "53ab064dd5ad4a12bb81957812ac806a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ab064dd5ad4a12bb81957812ac806a" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 37987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260806 22:43:59.813246 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.811246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.0.186.194:37987)
0806 22:43:59.811306 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:43:59.811308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:43:59.811317 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:43:59.811397 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:43:59.811397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:43:59.811411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:43:59.811458 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:43:59.811561 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.811981 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.812684 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:43:59.812853 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:43:59.812893 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:43:59.812988 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:43:59.813075 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:43:59.813128 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:43:59.813168 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:43:59.813182 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:43:59.814797 31102 raft_consensus.cc:1261] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 1 FOLLOWER]: Refusing update from remote peer 53ab064dd5ad4a12bb81957812ac806a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:59.815068 30686 raft_consensus.cc:1261] T 4e5b12234fc14e9e956bd225b079f40f P ef1df8ab856242ef9c9172bb8d8048cc [term 1 FOLLOWER]: Refusing update from remote peer 53ab064dd5ad4a12bb81957812ac806a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:43:59.815119 31201 consensus_queue.cc:1035] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c337c3c189bc4c678cd15fec8c8c7d53" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:59.815361 31204 consensus_queue.cc:1035] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1df8ab856242ef9c9172bb8d8048cc" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:43:59.818104 31208 mvcc.cc:205] Tried to move back new op lower bound from 7315686358279106560 to 7315686358241824768. Current Snapshot: MvccSnapshot[applied={T|T < 7315686358279106560}]
I20260806 22:43:59.820348 30455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:59.820442 30455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:43:59.821177 30455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22e7abadb8ee418e982ce90f0c4a5037: Sending DeleteTablet for 3 replicas of tablet 4e5b12234fc14e9e956bd225b079f40f
I20260806 22:43:59.821480 30862 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5b12234fc14e9e956bd225b079f40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:59 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:37782
I20260806 22:43:59.821525 30654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5b12234fc14e9e956bd225b079f40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:59 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:38998
I20260806 22:43:59.821859 31076 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5b12234fc14e9e956bd225b079f40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:43:59 UTC) from {username='kudu', principal='kudu/127.0.186.254@KRBTEST.COM'} at 127.0.0.1:39816
I20260806 22:43:59.821925 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 30553
I20260806 22:43:59.828747 31225 tablet_replica.cc:330] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: stopping tablet replica
I20260806 22:43:59.828855 31225 raft_consensus.cc:2229] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:43:59.828950 31225 raft_consensus.cc:2258] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:59.829401 31225 ts_tablet_manager.cc:1905] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:59.831045 31225 ts_tablet_manager.cc:1918] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:59.831123 31225 log.cc:1196] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/4e5b12234fc14e9e956bd225b079f40f
I20260806 22:43:59.831420 31225 ts_tablet_manager.cc:1939] T 4e5b12234fc14e9e956bd225b079f40f P 53ab064dd5ad4a12bb81957812ac806a: Deleting consensus metadata
I20260806 22:43:59.831542 31228 tablet_replica.cc:330] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: stopping tablet replica
I20260806 22:43:59.831688 31228 raft_consensus.cc:2229] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:43:59.831836 31228 raft_consensus.cc:2258] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:43:59.831808 30432 catalog_manager.cc:4841] TS 53ab064dd5ad4a12bb81957812ac806a (127.0.186.194:37987): tablet 4e5b12234fc14e9e956bd225b079f40f (table test-table [id=104bb6e7b438401ca531251bbaf68e8b]) successfully deleted
W20260806 22:43:59.834700 30433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:44465 (error 108)
I20260806 22:43:59.835191 31228 ts_tablet_manager.cc:1905] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:43:59.835206 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.835072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44465 (local address 127.0.0.1:39010)
0806 22:43:59.835128 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:43:59.835172 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:43:59.835444 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 30768
W20260806 22:43:59.835453 30433 catalog_manager.cc:4584] TS ef1df8ab856242ef9c9172bb8d8048cc (127.0.186.193:44465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e5b12234fc14e9e956bd225b079f40f: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44465: connect: Connection refused (error 111)
I20260806 22:43:59.837205 31228 ts_tablet_manager.cc:1918] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:43:59.837263 31228 log.cc:1196] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/4e5b12234fc14e9e956bd225b079f40f
I20260806 22:43:59.837487 31228 ts_tablet_manager.cc:1939] T 4e5b12234fc14e9e956bd225b079f40f P c337c3c189bc4c678cd15fec8c8c7d53: Deleting consensus metadata
I20260806 22:43:59.837898 30434 catalog_manager.cc:4841] TS c337c3c189bc4c678cd15fec8c8c7d53 (127.0.186.195:33909): tablet 4e5b12234fc14e9e956bd225b079f40f (table test-table [id=104bb6e7b438401ca531251bbaf68e8b]) successfully deleted
I20260806 22:43:59.845953 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 30975
I20260806 22:43:59.852504 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 30399
2026-08-06T22:43:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2044 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:43:59Z Disabled control of system clock
I20260806 22:43:59.885093 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35375
--webserver_interface=127.0.186.254
--webserver_port=0
--builtin_ntp_servers=127.0.186.212:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:35375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:43:59.965199 31235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:43:59.965402 31235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:43:59.965421 31235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:43:59.966559 31235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:43:59.966600 31235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:43:59.966612 31235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:43:59.966624 31235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:43:59.966646 31235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:43:59.968015 31235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:35375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:43:59.968254 31235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:43:59.968523 31235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:43:59.973069 31245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.973073 31246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:43:59.973086 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:43:59.973380 31235 server_base.cc:1034] running on GCE node
I20260806 22:43:59.973635 31235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:43:59.973917 31235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:43:59.975056 31235 hybrid_clock.cc:648] HybridClock initialized: now 1786056239975044 us; error 29 us; skew 500 ppm
I20260806 22:43:59.976322 31235 webserver.cc:460] Webserver started at http://127.0.186.254:36199/ using document root <none> and password file <none>
I20260806 22:43:59.976567 31235 fs_manager.cc:359] Metadata directory not provided
I20260806 22:43:59.976616 31235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:43:59.976742 31235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:43:59.977723 31235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "ee66b0c5e0c0456cab17c88c8dee7f94"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6fc34227b508a35d7dff229fc3c9839b"
server_key_iv: "57e52278140cd7205b7d7af536c82e41"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.978048 31235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "ee66b0c5e0c0456cab17c88c8dee7f94"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6fc34227b508a35d7dff229fc3c9839b"
server_key_iv: "57e52278140cd7205b7d7af536c82e41"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.979305 31235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:43:59.979995 31255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:43:59.980142 31235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:43:59.980224 31235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "ee66b0c5e0c0456cab17c88c8dee7f94"
format_stamp: "Formatted at 2026-08-06 22:43:59 on dist-test-slave-gjrz"
server_key: "6fc34227b508a35d7dff229fc3c9839b"
server_key_iv: "57e52278140cd7205b7d7af536c82e41"
server_key_version: "encryptionkey@0"
I20260806 22:43:59.980290 31235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:43:59.993780 31235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:43:59.994068 31235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:43:59.994199 31235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:43:59.998477 31235 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:35375
I20260806 22:43:59.998507 31332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:35375 every 8 connection(s)
I20260806 22:43:59.998804 31235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:43:59.999147 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 31235
I20260806 22:43:59.999253 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:43:59.999467 747 external_mini_cluster.cc:1428] Setting key 45e9680d9f22897757d508b5e9e3a9b1
I20260806 22:43:59.999799 31333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:00.002557 31333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: Bootstrap starting.
I20260806 22:44:00.003140 31333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:00.003443 31333 log.cc:824] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:00.004202 31333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: No bootstrap required, opened a new log
I20260806 22:44:00.004807 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:43:59.999933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.0.186.254:35375)
0806 22:44:00.000154 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.000161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.000185 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:44:00.000838 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.000846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.000908 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.001042 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:00.001264 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.002126 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.002934 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.003391 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.003516 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.004168 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:00.004172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:00.004172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:00.004351 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:00.004385 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:00.004397 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:00.004448 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.004543 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.004604 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260806 22:44:00.006170 31333 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } }
I20260806 22:44:00.006301 31333 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:00.006323 31333 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee66b0c5e0c0456cab17c88c8dee7f94, State: Initialized, Role: FOLLOWER
I20260806 22:44:00.006413 31333 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } }
I20260806 22:44:00.006489 31333 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:00.006516 31333 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:00.006547 31333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:00.007046 31333 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } }
I20260806 22:44:00.007136 31333 leader_election.cc:302] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee66b0c5e0c0456cab17c88c8dee7f94; no voters:
I20260806 22:44:00.007288 31333 leader_election.cc:288] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:00.007334 31339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:00.007494 31339 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [term 1 LEADER]: Becoming Leader. State: Replica: ee66b0c5e0c0456cab17c88c8dee7f94, State: Running, Role: LEADER
I20260806 22:44:00.007552 31333 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:00.007586 31339 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } }
I20260806 22:44:00.007977 31344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee66b0c5e0c0456cab17c88c8dee7f94. Latest consensus state: current_term: 1 leader_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } } }
I20260806 22:44:00.008087 31344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:00.007970 31340 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee66b0c5e0c0456cab17c88c8dee7f94" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 35375 } } }
I20260806 22:44:00.008267 31340 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:00.008492 31353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:00.009061 31353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:00.010438 31353 catalog_manager.cc:1347] Generated new cluster ID: 4df0d68992c346e8bca9c83811587ce6
I20260806 22:44:00.010485 31353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:00.019304 31353 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:00.019946 31353 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:00.023396 31353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: Generated new TSK 0
I20260806 22:44:00.023552 31353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:00.031654 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.193:0
--local_ip_for_outbound_sockets=127.0.186.193
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--builtin_ntp_servers=127.0.186.212:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:44:00.112967 31372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:00.113170 31372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:00.113200 31372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:00.114336 31372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:00.114377 31372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:00.114413 31372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.193
I20260806 22:44:00.115792 31372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.186.193
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.193
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:00.116027 31372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:00.116305 31372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:00.121073 31382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:00.121104 31388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:00.121099 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:00.121601 31372 server_base.cc:1034] running on GCE node
I20260806 22:44:00.121805 31372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:00.122063 31372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:00.123215 31372 hybrid_clock.cc:648] HybridClock initialized: now 1786056240123198 us; error 35 us; skew 500 ppm
I20260806 22:44:00.124440 31372 webserver.cc:460] Webserver started at http://127.0.186.193:46611/ using document root <none> and password file <none>
I20260806 22:44:00.124678 31372 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:00.124724 31372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:00.124850 31372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:00.125922 31372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data/instance:
uuid: "d2c8750d51644465a7994b14fc0cce5a"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "b142932467b42b4d9f85401f2fe4c3cb"
server_key_iv: "e44f272a619821306274925408560a55"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.126258 31372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal/instance:
uuid: "d2c8750d51644465a7994b14fc0cce5a"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "b142932467b42b4d9f85401f2fe4c3cb"
server_key_iv: "e44f272a619821306274925408560a55"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.127561 31372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:00.128396 31397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.128558 31372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:00.128648 31372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal
uuid: "d2c8750d51644465a7994b14fc0cce5a"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "b142932467b42b4d9f85401f2fe4c3cb"
server_key_iv: "e44f272a619821306274925408560a55"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.128718 31372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:00.147197 31372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:00.147516 31372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:00.147637 31372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:00.147838 31372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:00.148154 31372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:00.148188 31372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.148216 31372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:00.148240 31372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.153744 31372 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.193:44605
I20260806 22:44:00.153798 31561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.193:44605 every 8 connection(s)
I20260806 22:44:00.154088 31372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:00.157303 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 31372
I20260806 22:44:00.157408 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:00.157595 747 external_mini_cluster.cc:1428] Setting key 9b68b90e4d9e0167b5af6a3505cee9e1
I20260806 22:44:00.158667 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.194:0
--local_ip_for_outbound_sockets=127.0.186.194
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--builtin_ntp_servers=127.0.186.212:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:44:00.159353 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.154751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:54759 (local address 127.0.186.254:35375)
0806 22:44:00.154811 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.154812 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.155059 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:44:00.155635 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.155636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.155652 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.155726 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:00.156369 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.156865 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.157883 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.158147 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.158209 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.159101 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:00.159102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:00.159102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:00.159106 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:00.159122 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:00.159125 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:00.159161 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.159242 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.159289 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:44:00.159578 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.154749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35375 (local address 127.0.186.193:54759)
0806 22:44:00.154941 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.154980 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.155538 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.155780 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.155784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.155794 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:00.156283 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.156288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.156946 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.156947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.157796 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.157799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.158246 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.158247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.158417 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.158940 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:44:00.158941 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:00.158985 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:44:00.159012 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:00.159175 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:00.159180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:00.159182 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.159246 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.159374 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260806 22:44:00.160105 31562 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35375
I20260806 22:44:00.160200 31562 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:00.160391 31562 heartbeater.cc:511] Master 127.0.186.254:35375 requested a full tablet report, sending...
I20260806 22:44:00.160909 31281 ts_manager.cc:194] Registered new tserver with Master: d2c8750d51644465a7994b14fc0cce5a (127.0.186.193:44605)
I20260806 22:44:00.161746 31281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.193:54759
W20260806 22:44:00.239387 31569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:00.239573 31569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:00.239593 31569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:00.240700 31569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:00.240731 31569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:00.240754 31569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.194
I20260806 22:44:00.242079 31569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.186.194
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.194
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:00.242255 31569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:00.242493 31569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:00.247085 31581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:00.247169 31580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:00.247292 31569 server_base.cc:1034] running on GCE node
W20260806 22:44:00.247381 31584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:00.247581 31569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:00.247798 31569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:00.248939 31569 hybrid_clock.cc:648] HybridClock initialized: now 1786056240248918 us; error 36 us; skew 500 ppm
I20260806 22:44:00.250134 31569 webserver.cc:460] Webserver started at http://127.0.186.194:41109/ using document root <none> and password file <none>
I20260806 22:44:00.250357 31569 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:00.250404 31569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:00.250532 31569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:00.251570 31569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data/instance:
uuid: "df81160fec034622b8f11f0150ec7721"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "425ec7b0f9a16b184bbe142e4dedb703"
server_key_iv: "b3b5421f36029eaa340305655f3409eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.251885 31569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal/instance:
uuid: "df81160fec034622b8f11f0150ec7721"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "425ec7b0f9a16b184bbe142e4dedb703"
server_key_iv: "b3b5421f36029eaa340305655f3409eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.253038 31569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:00.253656 31592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.253808 31569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:00.253880 31569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal
uuid: "df81160fec034622b8f11f0150ec7721"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "425ec7b0f9a16b184bbe142e4dedb703"
server_key_iv: "b3b5421f36029eaa340305655f3409eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.253940 31569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:00.284915 31569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:00.285209 31569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:00.285338 31569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:00.285564 31569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:00.285880 31569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:00.285915 31569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.285944 31569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:00.285977 31569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.291388 31569 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.194:42699
I20260806 22:44:00.291450 31764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.194:42699 every 8 connection(s)
I20260806 22:44:00.291740 31569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:00.294626 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 31569
I20260806 22:44:00.294746 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:00.294972 747 external_mini_cluster.cc:1428] Setting key 6874ed9ad38b413261943e0467c79d29
I20260806 22:44:00.295989 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.186.195:0
--local_ip_for_outbound_sockets=127.0.186.195
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--builtin_ntp_servers=127.0.186.212:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:44:00.297233 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.292419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.194:50727 (local address 127.0.186.254:35375)
0806 22:44:00.292484 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.292486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.292723 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:44:00.293247 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.293248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.293259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.293322 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:00.294004 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.294504 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.295846 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.296037 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.296104 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.297005 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:00.297007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:00.297007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:00.297011 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:00.297026 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:00.297030 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:00.297070 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.297152 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.297175 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:44:00.297473 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.292392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35375 (local address 127.0.186.194:50727)
0806 22:44:00.292585 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.292626 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.293108 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.293395 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.293398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.293406 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:00.293904 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.293908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.294616 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.294617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.295754 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.295757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.296203 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.296204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.296355 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.296855 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:44:00.296855 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:00.296891 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:44:00.296919 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:00.297083 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:00.297087 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:00.297090 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.297151 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.297260 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260806 22:44:00.298106 31765 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35375
I20260806 22:44:00.298206 31765 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:00.298395 31765 heartbeater.cc:511] Master 127.0.186.254:35375 requested a full tablet report, sending...
I20260806 22:44:00.298765 31281 ts_manager.cc:194] Registered new tserver with Master: df81160fec034622b8f11f0150ec7721 (127.0.186.194:42699)
I20260806 22:44:00.299108 31281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.194:50727
W20260806 22:44:00.378013 31773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:00.378191 31773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:00.378211 31773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:00.379354 31773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:00.379392 31773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:00.379421 31773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.186.195
I20260806 22:44:00.380751 31773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.186.195
--webserver_port=0
--tserver_master_addrs=127.0.186.254:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.186.195
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:00.380956 31773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:00.381218 31773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:00.385950 31781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:00.385952 31782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:00.386075 31773 server_base.cc:1034] running on GCE node
W20260806 22:44:00.386160 31786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:00.386358 31773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:00.386562 31773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:00.387701 31773 hybrid_clock.cc:648] HybridClock initialized: now 1786056240387687 us; error 27 us; skew 500 ppm
I20260806 22:44:00.388938 31773 webserver.cc:460] Webserver started at http://127.0.186.195:45933/ using document root <none> and password file <none>
I20260806 22:44:00.389183 31773 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:00.389233 31773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:00.389358 31773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:00.390348 31773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data/instance:
uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "78319456b82b6b850519377fc653cb6a"
server_key_iv: "9e755278b184098b865f0f54b35f4918"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.390667 31773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal/instance:
uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "78319456b82b6b850519377fc653cb6a"
server_key_iv: "9e755278b184098b865f0f54b35f4918"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.391940 31773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:00.392673 31793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.392844 31773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:00.392926 31773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal
uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc"
format_stamp: "Formatted at 2026-08-06 22:44:00 on dist-test-slave-gjrz"
server_key: "78319456b82b6b850519377fc653cb6a"
server_key_iv: "9e755278b184098b865f0f54b35f4918"
server_key_version: "encryptionkey@0"
I20260806 22:44:00.392987 31773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:00.418154 31773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:00.418423 31773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:00.418545 31773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:00.418746 31773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:00.419071 31773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:00.419106 31773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.419132 31773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:00.419149 31773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:00.424706 31773 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.195:45701
I20260806 22:44:00.424763 31979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.195:45701 every 8 connection(s)
I20260806 22:44:00.425094 31773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:00.429979 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.425756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.195:60509 (local address 127.0.186.254:35375)
0806 22:44:00.425818 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.425820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.426056 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:44:00.426529 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.426531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.426544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.426598 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:00.427236 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.427765 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.428713 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.428913 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.428968 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.429722 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:00.429724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:00.429724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:00.429729 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:00.429745 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:00.429749 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:00.429794 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.429873 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.429900 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:44:00.430178 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.425739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:35375 (local address 127.0.186.195:60509)
0806 22:44:00.425927 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.425960 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.426435 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.426666 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.426669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.426678 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:00.427148 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.427154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.427852 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.427854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.428628 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.428630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.428982 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.428983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.429109 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.429587 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:44:00.429588 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:00.429616 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:44:00.429639 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:00.429800 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:00.429805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:00.429808 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.429869 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.429970 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260806 22:44:00.430648 31981 heartbeater.cc:347] Connected to a master server at 127.0.186.254:35375
I20260806 22:44:00.430738 31981 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:00.430888 31981 heartbeater.cc:511] Master 127.0.186.254:35375 requested a full tablet report, sending...
I20260806 22:44:00.431249 31281 ts_manager.cc:194] Registered new tserver with Master: 32c91cfa7e8e4bcc94dc0ee900c046cc (127.0.186.195:45701)
I20260806 22:44:00.431681 31281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.186.195:60509
I20260806 22:44:00.432119 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 31773
I20260806 22:44:00.432206 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:00.432385 747 external_mini_cluster.cc:1428] Setting key 521bbe7c920141af2f331d55ec79e140
I20260806 22:44:00.433159 747 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:00.436787 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.434414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.0.186.254:35375)
0806 22:44:00.434448 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.434450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.434566 (+ 116us) server_negotiation.cc:407] Connection header received
0806 22:44:00.434604 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.434605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.434612 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.434656 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:00.434762 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.435194 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.435936 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.436114 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.436158 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.436632 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:00.436633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:00.436633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:00.436637 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:00.436649 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:00.436653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:00.436688 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.436721 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.436738 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:44:00.438205 31281 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:00.439757 31281 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:00.445474 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.0.186.195:45701)
0806 22:44:00.441852 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.441856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.442029 (+ 173us) server_negotiation.cc:407] Connection header received
0806 22:44:00.442113 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.442115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.442130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.442207 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:00.442347 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443433 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.444628 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445104 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445168 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445255 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.445365 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.445400 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260806 22:44:00.445464 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45701 (local address 127.0.0.1:57748)
0806 22:44:00.441922 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.441959 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.442034 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.442210 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.442214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.442215 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:00.442275 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.442277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443528 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.443529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.444570 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.444572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445180 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.445262 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445277 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.445340 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.445392 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":8,"thread_start_us":50,"threads_started":1}
I20260806 22:44:00.445744 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44605 (local address 127.0.0.1:32878)
0806 22:44:00.442144 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.442155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.442239 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.442471 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.442472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.442474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:00.442515 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.442516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443696 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.443697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.444595 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.444598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445517 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.445586 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445591 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.445637 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.445674 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":170,"threads_started":1}
I20260806 22:44:00.445911 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.0.186.194:42699)
0806 22:44:00.442054 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.442057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.442068 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:00.442144 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.442147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.442161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.442228 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:00.442411 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443185 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.444581 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445286 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445357 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445445 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.445789 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.445830 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260806 22:44:00.446040 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.0.186.193:44605)
0806 22:44:00.441963 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.441966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.442210 (+ 244us) server_negotiation.cc:407] Connection header received
0806 22:44:00.442315 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.442318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.442333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.442402 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:00.442591 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443628 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.444677 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445345 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445399 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445481 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.445911 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.445956 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260806 22:44:00.446125 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.441638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42699 (local address 127.0.0.1:44446)
0806 22:44:00.442001 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.442011 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.442073 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.442291 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.442292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.442294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:00.442323 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.442324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.443280 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.443281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.444504 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.444507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.445588 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.445589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.445649 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.445654 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.445703 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.445744 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":215,"threads_started":1}
I20260806 22:44:00.446206 31869 tablet_service.cc:1467] Processing CreateTablet for tablet 102a0c554c09404d8214c80669021519 (DEFAULT_TABLE table=test-table [id=ed8593291347408da301341fd687f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:00.446568 31869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a0c554c09404d8214c80669021519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:00.446653 31463 tablet_service.cc:1467] Processing CreateTablet for tablet 102a0c554c09404d8214c80669021519 (DEFAULT_TABLE table=test-table [id=ed8593291347408da301341fd687f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:00.446769 31663 tablet_service.cc:1467] Processing CreateTablet for tablet 102a0c554c09404d8214c80669021519 (DEFAULT_TABLE table=test-table [id=ed8593291347408da301341fd687f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:00.446957 31463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a0c554c09404d8214c80669021519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:00.447059 31663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a0c554c09404d8214c80669021519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:00.449717 32005 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Bootstrap starting.
I20260806 22:44:00.449781 32006 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Bootstrap starting.
I20260806 22:44:00.450373 32006 tablet_bootstrap.cc:654] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:00.450367 32008 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: Bootstrap starting.
I20260806 22:44:00.450395 32005 tablet_bootstrap.cc:654] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:00.450700 32005 log.cc:824] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:00.450700 32006 log.cc:824] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:00.450924 32008 tablet_bootstrap.cc:654] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:00.451253 32008 log.cc:824] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:00.451483 32005 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: No bootstrap required, opened a new log
I20260806 22:44:00.451557 32005 ts_tablet_manager.cc:1397] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:00.451699 32006 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: No bootstrap required, opened a new log
I20260806 22:44:00.451759 32006 ts_tablet_manager.cc:1397] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:00.451968 32008 tablet_bootstrap.cc:492] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: No bootstrap required, opened a new log
I20260806 22:44:00.452033 32008 ts_tablet_manager.cc:1397] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:44:00.453858 32008 raft_consensus.cc:348] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.453989 32008 raft_consensus.cc:374] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:00.454027 32008 raft_consensus.cc:729] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c8750d51644465a7994b14fc0cce5a, State: Initialized, Role: FOLLOWER
I20260806 22:44:00.454053 32005 raft_consensus.cc:348] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.454053 32006 raft_consensus.cc:348] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.454159 32008 consensus_queue.cc:260] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.454212 32006 raft_consensus.cc:374] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:00.454212 32005 raft_consensus.cc:374] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:00.454257 32005 raft_consensus.cc:729] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df81160fec034622b8f11f0150ec7721, State: Initialized, Role: FOLLOWER
I20260806 22:44:00.454257 32006 raft_consensus.cc:729] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c91cfa7e8e4bcc94dc0ee900c046cc, State: Initialized, Role: FOLLOWER
I20260806 22:44:00.454396 32008 ts_tablet_manager.cc:1428] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:44:00.454365 32006 consensus_queue.cc:260] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.454411 32005 consensus_queue.cc:260] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.454510 31562 heartbeater.cc:503] Master 127.0.186.254:35375 was elected leader, sending a full tablet report...
I20260806 22:44:00.454631 32006 ts_tablet_manager.cc:1428] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:00.454685 31981 heartbeater.cc:503] Master 127.0.186.254:35375 was elected leader, sending a full tablet report...
I20260806 22:44:00.454631 32005 ts_tablet_manager.cc:1428] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:44:00.455014 31765 heartbeater.cc:503] Master 127.0.186.254:35375 was elected leader, sending a full tablet report...
I20260806 22:44:00.486430 32013 raft_consensus.cc:482] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:00.486531 32013 raft_consensus.cc:504] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.486807 32013 leader_election.cc:288] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c91cfa7e8e4bcc94dc0ee900c046cc (127.0.186.195:45701), df81160fec034622b8f11f0150ec7721 (127.0.186.194:42699)
I20260806 22:44:00.489775 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.487134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:44607 (local address 127.0.186.194:42699)
0806 22:44:00.487180 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.487182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.487190 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:00.487215 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.487216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.487227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.487301 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:00.487424 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.488196 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489095 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.489464 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489517 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.489524 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.489676 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.489713 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:44:00.489804 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.486941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.194:42699 (local address 127.0.186.193:44607)
0806 22:44:00.487013 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.487027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.487082 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.487312 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.487313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.487314 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:00.487374 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.487375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.488278 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.488279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.489015 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.489017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.489545 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.489614 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.489617 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.489665 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.489703 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:44:00.489811 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.487025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.186.193:36549 (local address 127.0.186.195:45701)
0806 22:44:00.487080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.487082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.487192 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:44:00.487241 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.487241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.487249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.487318 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:00.487514 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.488194 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489155 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.489514 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489584 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.489591 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.489717 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.489755 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:44:00.489976 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.486941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.195:45701 (local address 127.0.186.193:36549)
0806 22:44:00.487117 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.487127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:00.487189 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:00.487428 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:00.487428 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:00.487429 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:00.487466 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.487467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.488315 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.488316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.489061 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:00.489063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.489591 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.489592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:00.489649 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.489653 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:00.489704 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:44:00.489740 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":13,"thread_start_us":105,"threads_started":1}
I20260806 22:44:00.490104 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a0c554c09404d8214c80669021519" candidate_uuid: "d2c8750d51644465a7994b14fc0cce5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df81160fec034622b8f11f0150ec7721" is_pre_election: true
I20260806 22:44:00.490247 31696 raft_consensus.cc:2454] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2c8750d51644465a7994b14fc0cce5a in term 0.
I20260806 22:44:00.490273 31909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a0c554c09404d8214c80669021519" candidate_uuid: "d2c8750d51644465a7994b14fc0cce5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" is_pre_election: true
I20260806 22:44:00.490410 31909 raft_consensus.cc:2454] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2c8750d51644465a7994b14fc0cce5a in term 0.
I20260806 22:44:00.490463 31399 leader_election.cc:302] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2c8750d51644465a7994b14fc0cce5a, df81160fec034622b8f11f0150ec7721; no voters:
I20260806 22:44:00.490574 32013 raft_consensus.cc:2781] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:00.490628 32013 raft_consensus.cc:482] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:00.490648 32013 raft_consensus.cc:3037] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:00.491370 32013 raft_consensus.cc:504] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.491564 32013 leader_election.cc:288] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [CANDIDATE]: Term 1 election: Requested vote from peers 32c91cfa7e8e4bcc94dc0ee900c046cc (127.0.186.195:45701), df81160fec034622b8f11f0150ec7721 (127.0.186.194:42699)
I20260806 22:44:00.491626 31909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a0c554c09404d8214c80669021519" candidate_uuid: "d2c8750d51644465a7994b14fc0cce5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc"
I20260806 22:44:00.491707 31909 raft_consensus.cc:3037] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:00.491709 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a0c554c09404d8214c80669021519" candidate_uuid: "d2c8750d51644465a7994b14fc0cce5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df81160fec034622b8f11f0150ec7721"
I20260806 22:44:00.491772 31696 raft_consensus.cc:3037] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:00.492563 31909 raft_consensus.cc:2454] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c8750d51644465a7994b14fc0cce5a in term 1.
I20260806 22:44:00.492592 31696 raft_consensus.cc:2454] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c8750d51644465a7994b14fc0cce5a in term 1.
I20260806 22:44:00.492718 31399 leader_election.cc:302] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2c8750d51644465a7994b14fc0cce5a, df81160fec034622b8f11f0150ec7721; no voters:
I20260806 22:44:00.492827 32013 raft_consensus.cc:2781] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:00.492964 32013 raft_consensus.cc:686] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 1 LEADER]: Becoming Leader. State: Replica: d2c8750d51644465a7994b14fc0cce5a, State: Running, Role: LEADER
I20260806 22:44:00.493059 32013 consensus_queue.cc:237] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } }
I20260806 22:44:00.493820 31280 catalog_manager.cc:5495] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a reported cstate change: term changed from 0 to 1, leader changed from <none> to d2c8750d51644465a7994b14fc0cce5a (127.0.186.193). New cstate: current_term: 1 leader_uuid: "d2c8750d51644465a7994b14fc0cce5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c8750d51644465a7994b14fc0cce5a" member_type: VOTER last_known_addr { host: "127.0.186.193" port: 44605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:00.501834 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.499755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.0.186.193:44605)
0806 22:44:00.499800 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:00.499801 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:00.499808 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:00.499838 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:00.499839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:00.499853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:00.499908 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:00.500000 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.500436 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.501220 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:00.501387 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:00.501428 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:00.501599 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:00.501684 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:00.501734 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:00.501751 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 22:44:00.501766 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:44:00.503418 31696 raft_consensus.cc:1261] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 1 FOLLOWER]: Refusing update from remote peer d2c8750d51644465a7994b14fc0cce5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:00.503418 31909 raft_consensus.cc:1261] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 1 FOLLOWER]: Refusing update from remote peer d2c8750d51644465a7994b14fc0cce5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:00.503739 32018 consensus_queue.cc:1035] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df81160fec034622b8f11f0150ec7721" member_type: VOTER last_known_addr { host: "127.0.186.194" port: 42699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:00.503858 32013 consensus_queue.cc:1035] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c91cfa7e8e4bcc94dc0ee900c046cc" member_type: VOTER last_known_addr { host: "127.0.186.195" port: 45701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:00.506125 32023 mvcc.cc:205] Tried to move back new op lower bound from 7315686361099644928 to 7315686361060171776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686361099644928}]
I20260806 22:44:00.506623 32024 mvcc.cc:205] Tried to move back new op lower bound from 7315686361099644928 to 7315686361060171776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686361099644928}]
I20260806 22:44:00.506646 32025 mvcc.cc:205] Tried to move back new op lower bound from 7315686361099644928 to 7315686361060171776. Current Snapshot: MvccSnapshot[applied={T|T < 7315686361099644928}]
I20260806 22:44:00.508113 31280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:00.508217 31280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:00.508889 31280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee66b0c5e0c0456cab17c88c8dee7f94: Sending DeleteTablet for 3 replicas of tablet 102a0c554c09404d8214c80669021519
I20260806 22:44:00.509202 31463 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a0c554c09404d8214c80669021519 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:00 UTC) from {username='slave'} at 127.0.0.1:32878
I20260806 22:44:00.509195 31663 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a0c554c09404d8214c80669021519 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:00 UTC) from {username='slave'} at 127.0.0.1:44446
I20260806 22:44:00.509212 31869 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a0c554c09404d8214c80669021519 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:00 UTC) from {username='slave'} at 127.0.0.1:57748
I20260806 22:44:00.509426 32037 tablet_replica.cc:330] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a: stopping tablet replica
I20260806 22:44:00.509537 32037 raft_consensus.cc:2229] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:00.509651 32037 raft_consensus.cc:2258] T 102a0c554c09404d8214c80669021519 P d2c8750d51644465a7994b14fc0cce5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:00.509667 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 31372
I20260806 22:44:00.520846 32036 tablet_replica.cc:330] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: stopping tablet replica
I20260806 22:44:00.520967 32036 raft_consensus.cc:2229] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:00.520977 32034 tablet_replica.cc:330] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: stopping tablet replica
I20260806 22:44:00.521091 32036 raft_consensus.cc:2258] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:00.521672 32034 raft_consensus.cc:2229] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:00.521909 32034 raft_consensus.cc:2258] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:00.522831 32036 ts_tablet_manager.cc:1905] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:00.524893 32034 ts_tablet_manager.cc:1905] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:00.525650 32036 ts_tablet_manager.cc:1918] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:00.525729 32036 log.cc:1196] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-1/wal/wals/102a0c554c09404d8214c80669021519
I20260806 22:44:00.525970 32036 ts_tablet_manager.cc:1939] T 102a0c554c09404d8214c80669021519 P df81160fec034622b8f11f0150ec7721: Deleting consensus metadata
I20260806 22:44:00.526185 32034 ts_tablet_manager.cc:1918] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:00.526252 32034 log.cc:1196] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Deleting WAL directory at /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055997241161-747-0/minicluster-data/ts-2/wal/wals/102a0c554c09404d8214c80669021519
I20260806 22:44:00.526428 31259 catalog_manager.cc:4841] TS df81160fec034622b8f11f0150ec7721 (127.0.186.194:42699): tablet 102a0c554c09404d8214c80669021519 (table test-table [id=ed8593291347408da301341fd687f421]) successfully deleted
I20260806 22:44:00.526510 32034 ts_tablet_manager.cc:1939] T 102a0c554c09404d8214c80669021519 P 32c91cfa7e8e4bcc94dc0ee900c046cc: Deleting consensus metadata
I20260806 22:44:00.526875 31264 catalog_manager.cc:4841] TS 32c91cfa7e8e4bcc94dc0ee900c046cc (127.0.186.195:45701): tablet 102a0c554c09404d8214c80669021519 (table test-table [id=ed8593291347408da301341fd687f421]) successfully deleted
W20260806 22:44:00.529155 31263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.186.193:44605 (error 108)
I20260806 22:44:00.529354 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 31569
I20260806 22:44:00.529652 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:00.529533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.193:44605 (local address 127.0.0.1:32890)
0806 22:44:00.529584 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:00.529621 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:44:00.529752 31263 catalog_manager.cc:4584] TS d2c8750d51644465a7994b14fc0cce5a (127.0.186.193:44605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 102a0c554c09404d8214c80669021519: Network error: Client connection negotiation failed: client connection to 127.0.186.193:44605: connect: Connection refused (error 111)
I20260806 22:44:00.535337 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 31773
I20260806 22:44:00.553395 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 31235
2026-08-06T22:44:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (699 ms)
[----------] 6 tests from EncryptionPolicyTest (8628 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:00 dist-test-slave-gjrz krb5kdc[32042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:00 dist-test-slave-gjrz krb5kdc[32042](info): set up 2 sockets
Aug 06 22:44:00 dist-test-slave-gjrz krb5kdc[32042](info): commencing operation
krb5kdc: starting...
W20260806 22:44:01.679171 747 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:01 dist-test-slave-gjrz krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:01.793799 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:01.873315 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:01.873504 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:01.873525 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:01.874626 32058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:01.874666 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:01.874678 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:01.874689 32058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:01.874701 32058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:01.876178 32058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:01.876398 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:01.876619 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:01.881232 32065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:01.881402 32058 server_base.cc:1034] running on GCE node
W20260806 22:44:01.881242 32069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:01.881259 32071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:01.881783 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:01.882099 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:01.883245 32058 hybrid_clock.cc:648] HybridClock initialized: now 1786056241883226 us; error 33 us; skew 500 ppm
Aug 06 22:44:01 dist-test-slave-gjrz krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:01.885648 32058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:01.886080 32058 webserver.cc:460] Webserver started at http://127.0.0.1:44377/ using document root <none> and password file <none>
I20260806 22:44:01.886301 32058 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:01.886349 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:01.886466 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:01.887501 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460"
format_stamp: "Formatted at 2026-08-06 22:44:01 on dist-test-slave-gjrz"
server_key: "04dbfd4db2569c65ae7a1da2ccaca9e0"
server_key_iv: "32910ffc6d559dde34a66f083a905f80"
server_key_version: "encryptionkey@0"
I20260806 22:44:01.887814 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460"
format_stamp: "Formatted at 2026-08-06 22:44:01 on dist-test-slave-gjrz"
server_key: "04dbfd4db2569c65ae7a1da2ccaca9e0"
server_key_iv: "32910ffc6d559dde34a66f083a905f80"
server_key_version: "encryptionkey@0"
I20260806 22:44:01.889067 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:01.889729 32083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:01.889895 32058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:01.889976 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460"
format_stamp: "Formatted at 2026-08-06 22:44:01 on dist-test-slave-gjrz"
server_key: "04dbfd4db2569c65ae7a1da2ccaca9e0"
server_key_iv: "32910ffc6d559dde34a66f083a905f80"
server_key_version: "encryptionkey@0"
I20260806 22:44:01.890044 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:01.908325 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:01.910336 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:01.910459 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:01.914179 32058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40299
I20260806 22:44:01.914238 32160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40299 every 8 connection(s)
I20260806 22:44:01.914528 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:01.915480 32162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:01.917729 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460: Bootstrap starting.
I20260806 22:44:01.918290 32162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:01.918362 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 32058
I20260806 22:44:01.918468 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:01.918589 32162 log.cc:824] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:01.918661 747 external_mini_cluster.cc:1428] Setting key 2ef1d767987cb64f84503788e68683ca
I20260806 22:44:01.919404 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460: No bootstrap required, opened a new log
I20260806 22:44:01.921749 32162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } }
I20260806 22:44:01.921936 32162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:01.921972 32162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb9e0196c8a4a08a2fc7263c4cf3460, State: Initialized, Role: FOLLOWER
I20260806 22:44:01.922103 32162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } }
I20260806 22:44:01.922179 32162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:01.922212 32162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:01.922255 32162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:01.922869 32162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } }
I20260806 22:44:01.922968 32162 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb9e0196c8a4a08a2fc7263c4cf3460; no voters:
I20260806 22:44:01.923134 32162 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:01.923180 32171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:01.923413 32171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [term 1 LEADER]: Becoming Leader. State: Replica: 2eb9e0196c8a4a08a2fc7263c4cf3460, State: Running, Role: LEADER
I20260806 22:44:01.923409 32162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:01.923523 32171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } }
I20260806 22:44:01.923830 32172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } } }
I20260806 22:44:01.923913 32172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:01.924110 32178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:01.924633 32178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:01 dist-test-slave-gjrz krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:44:01.925501 32173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb9e0196c8a4a08a2fc7263c4cf3460. Latest consensus state: current_term: 1 leader_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb9e0196c8a4a08a2fc7263c4cf3460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40299 } } }
I20260806 22:44:01.926450 32173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:01.926820 32178 catalog_manager.cc:1347] Generated new cluster ID: f06d4a01f8c345b49591eaf40ff7f266
I20260806 22:44:01.926874 32178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:01.929409 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:01.919193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56425 (local address 127.0.0.1:40299)
0806 22:44:01.919360 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:01.919370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:01.919403 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:44:01.919840 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:01.919847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:01.919902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:01.920011 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:01.920778 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:01.921665 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:01.922671 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:01.923142 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:01.923256 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:01.926372 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:01.926376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:01.926376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:01.926389 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:01.928011 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:01.928364 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:01.928367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:01.928368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:01.928449 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:01.928648 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:01.928649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:01.928649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:01.928865 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:01.929019 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:01.929164 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:44:01.929261 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260806 22:44:01.930619 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:01.937247 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:01.931916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46681 (local address 127.0.0.1:40299)
0806 22:44:01.931954 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:01.931956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:01.932030 (+ 74us) server_negotiation.cc:407] Connection header received
0806 22:44:01.932085 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:01.932086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:01.932095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:01.932138 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:01.932580 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:01.933043 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:01.933688 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:01.933924 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:01.933976 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:01.934989 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:01.934991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:01.934991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:01.934997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:01.936213 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:01.936533 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:01.936541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:01.936542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:01.936599 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:01.936786 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:01.936787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:01.936787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:01.936886 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:01.936971 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:01.937104 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:44:01.937163 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:01.938666 32178 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:01.939343 32178 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:01.944940 32178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eb9e0196c8a4a08a2fc7263c4cf3460: Generated new TSK 0
I20260806 22:44:01.945097 32178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:01.947085 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 32058
W20260806 22:44:01.960966 32039 connection.cc:535] client connection to 127.0.0.1:40299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40299): Connection reset by peer (error 104)
2026-08-06T22:44:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:02 dist-test-slave-gjrz krb5kdc[32208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:02 dist-test-slave-gjrz krb5kdc[32208](info): set up 2 sockets
Aug 06 22:44:02 dist-test-slave-gjrz krb5kdc[32208](info): commencing operation
krb5kdc: starting...
W20260806 22:44:03.068557 747 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:03 dist-test-slave-gjrz krb5kdc[32208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:03.184114 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:03.264143 32224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:03.264338 32224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:03.264358 32224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:03.265489 32224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:03.265529 32224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:03.265542 32224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:03.265553 32224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:03.265564 32224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:03.267095 32224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:03.267354 32224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:03.267565 32224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:03.272169 32233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:03.272177 32238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:03.272177 32234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:03.272506 32224 server_base.cc:1034] running on GCE node
I20260806 22:44:03.272709 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:03.272979 32224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:03.274124 32224 hybrid_clock.cc:648] HybridClock initialized: now 1786056243274104 us; error 35 us; skew 500 ppm
Aug 06 22:44:03 dist-test-slave-gjrz krb5kdc[32208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:03.276685 32224 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:03.277117 32224 webserver.cc:460] Webserver started at http://127.0.0.1:43791/ using document root <none> and password file <none>
I20260806 22:44:03.277312 32224 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:03.277357 32224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:03.277490 32224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:03.278554 32224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "9ff2d969a796472a8f6fc20e6a7913ec"
format_stamp: "Formatted at 2026-08-06 22:44:03 on dist-test-slave-gjrz"
server_key: "abeab661bfa64493811580167abb898a"
server_key_iv: "171ecc44283caafa7b295dd3858af2eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:03.278891 32224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "9ff2d969a796472a8f6fc20e6a7913ec"
format_stamp: "Formatted at 2026-08-06 22:44:03 on dist-test-slave-gjrz"
server_key: "abeab661bfa64493811580167abb898a"
server_key_iv: "171ecc44283caafa7b295dd3858af2eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:03.280234 32224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:44:03.280965 32247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:03.281116 32224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:03.281203 32224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "9ff2d969a796472a8f6fc20e6a7913ec"
format_stamp: "Formatted at 2026-08-06 22:44:03 on dist-test-slave-gjrz"
server_key: "abeab661bfa64493811580167abb898a"
server_key_iv: "171ecc44283caafa7b295dd3858af2eb"
server_key_version: "encryptionkey@0"
I20260806 22:44:03.281268 32224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:03.290704 32224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:03.292447 32224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:03.292567 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:03.296319 32224 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35153
I20260806 22:44:03.296353 32323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35153 every 8 connection(s)
I20260806 22:44:03.296650 32224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:03.297427 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:03.298647 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 32224
I20260806 22:44:03.298744 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:03.298910 747 external_mini_cluster.cc:1428] Setting key 81c09c4b958c6eb9ab3faa3c5091a3a0
I20260806 22:44:03.300043 32324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec: Bootstrap starting.
I20260806 22:44:03.300735 32324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:03.301050 32324 log.cc:824] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:03.301767 32324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec: No bootstrap required, opened a new log
I20260806 22:44:03.303834 32324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260806 22:44:03.304008 32324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:03.304056 32324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff2d969a796472a8f6fc20e6a7913ec, State: Initialized, Role: FOLLOWER
I20260806 22:44:03.304183 32324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260806 22:44:03.304265 32324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:03.304297 32324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:03.304344 32324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:03.304719 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:03.299414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55409 (local address 127.0.0.1:35153)
0806 22:44:03.299597 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:03.299607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:03.299642 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:44:03.300145 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:03.300154 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:03.300220 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:03.300370 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:03.300534 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:03.301762 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:03.302555 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:03.303170 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:03.303306 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:03.303960 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:03.303963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:03.303964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:03.304156 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:03.304223 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:03.304240 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:03.304328 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:03.304496 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:44:03.304530 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260806 22:44:03.304898 32324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260806 22:44:03.305148 32324 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff2d969a796472a8f6fc20e6a7913ec; no voters:
I20260806 22:44:03.305322 32324 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:03.305353 32334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:03.305516 32334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [term 1 LEADER]: Becoming Leader. State: Replica: 9ff2d969a796472a8f6fc20e6a7913ec, State: Running, Role: LEADER
I20260806 22:44:03.305656 32324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:03.305656 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:03.305656 32334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260806 22:44:03.306034 32334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff2d969a796472a8f6fc20e6a7913ec. Latest consensus state: current_term: 1 leader_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } } }
I20260806 22:44:03.306245 32334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:03.306180 32336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff2d969a796472a8f6fc20e6a7913ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } } }
I20260806 22:44:03.306658 32336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:03.306895 32345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:03.307446 32345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:03.308876 32345 catalog_manager.cc:1347] Generated new cluster ID: 97ea58b204ca4931b36b450d4ee9a972
I20260806 22:44:03.308916 32345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:03.318765 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:03.307137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56627 (local address 127.0.0.1:35153)
0806 22:44:03.307178 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:03.307180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:03.315457 (+ 8277us) server_negotiation.cc:407] Connection header received
0806 22:44:03.315518 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:03.315519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:03.315530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:03.315588 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:03.315730 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:03.316464 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:03.317517 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:03.317821 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:03.317872 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:03.318527 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:03.318529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:03.318529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:03.318534 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:03.318548 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:03.318552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:03.318601 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:03.318681 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:44:03.318691 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:44:03.332386 32345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:03.332872 32345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:03.340770 32345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff2d969a796472a8f6fc20e6a7913ec: Generated new TSK 0
I20260806 22:44:03.340926 32345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:03.344694 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 32224
W20260806 22:44:03.364921 32206 connection.cc:535] client connection to 127.0.0.1:35153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35153): Connection reset by peer (error 104)
2026-08-06T22:44:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:03 dist-test-slave-gjrz krb5kdc[32369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:03 dist-test-slave-gjrz krb5kdc[32369](info): set up 2 sockets
Aug 06 22:44:03 dist-test-slave-gjrz krb5kdc[32369](info): commencing operation
krb5kdc: starting...
W20260806 22:44:04.493335 747 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:04 dist-test-slave-gjrz krb5kdc[32369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:04.607007 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:04.686101 32389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:04.686291 32389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:04.686322 32389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:04.687444 32389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:04.687482 32389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:04.687501 32389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:04.687516 32389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:04.687532 32389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:04.688947 32389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38111
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:04.689173 32389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:04.689365 32389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:04.693884 32389 server_base.cc:1034] running on GCE node
W20260806 22:44:04.693869 32395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:04.693857 32399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:04.693866 32396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:04.694274 32389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:04.694552 32389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:04.695708 32389 hybrid_clock.cc:648] HybridClock initialized: now 1786056244695693 us; error 28 us; skew 500 ppm
Aug 06 22:44:04 dist-test-slave-gjrz krb5kdc[32369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:04.701308 32389 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:04.701740 32389 webserver.cc:460] Webserver started at http://127.0.0.1:33741/ using document root <none> and password file <none>
I20260806 22:44:04.701942 32389 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:04.701987 32389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:04.702112 32389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:04.703070 32389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "93643676575643ac8c7959369dbbb9af"
format_stamp: "Formatted at 2026-08-06 22:44:04 on dist-test-slave-gjrz"
server_key: "1512dc5ff58b8744ec5b312fdf192d7b"
server_key_iv: "2471625eb72e82b0c6f63f9b05fff9ef"
server_key_version: "encryptionkey@0"
I20260806 22:44:04.703419 32389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "93643676575643ac8c7959369dbbb9af"
format_stamp: "Formatted at 2026-08-06 22:44:04 on dist-test-slave-gjrz"
server_key: "1512dc5ff58b8744ec5b312fdf192d7b"
server_key_iv: "2471625eb72e82b0c6f63f9b05fff9ef"
server_key_version: "encryptionkey@0"
I20260806 22:44:04.704634 32389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:04.705343 32409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:04.705511 32389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:04.705602 32389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "93643676575643ac8c7959369dbbb9af"
format_stamp: "Formatted at 2026-08-06 22:44:04 on dist-test-slave-gjrz"
server_key: "1512dc5ff58b8744ec5b312fdf192d7b"
server_key_iv: "2471625eb72e82b0c6f63f9b05fff9ef"
server_key_version: "encryptionkey@0"
I20260806 22:44:04.705664 32389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:04.708913 32389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:04.710677 32389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:04.710800 32389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:04.714593 32389 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38111
I20260806 22:44:04.714632 32477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38111 every 8 connection(s)
I20260806 22:44:04.714908 32389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:04.715863 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:04.717990 32480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af: Bootstrap starting.
I20260806 22:44:04.718537 32480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:04.718807 32480 log.cc:824] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:04.719452 32480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af: No bootstrap required, opened a new log
I20260806 22:44:04.721123 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 32389
I20260806 22:44:04.721045 32480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } }
I20260806 22:44:04.721174 32480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:04.721204 32480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93643676575643ac8c7959369dbbb9af, State: Initialized, Role: FOLLOWER
I20260806 22:44:04.721235 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:04.721336 32480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } }
I20260806 22:44:04.721406 747 external_mini_cluster.cc:1428] Setting key 3f38f675dfa1ad6ec6711b05f5330751
I20260806 22:44:04.721412 32480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:04.721446 32480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:04.721493 32480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:04.722433 32480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } }
I20260806 22:44:04.722549 32480 leader_election.cc:302] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93643676575643ac8c7959369dbbb9af; no voters:
I20260806 22:44:04.722683 32480 leader_election.cc:288] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:04.722734 32487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:04.722901 32487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [term 1 LEADER]: Becoming Leader. State: Replica: 93643676575643ac8c7959369dbbb9af, State: Running, Role: LEADER
I20260806 22:44:04.722920 32480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:04.723053 32487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } }
I20260806 22:44:04.723439 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:04.721835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43407 (local address 127.0.0.1:38111)
0806 22:44:04.722056 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:04.722064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:04.722089 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:44:04.722178 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:04.722183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:04.722205 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:04.722307 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:04.722915 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:04.722929 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:04.722929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:04.722934 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:04.722980 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:04.722995 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:04.723061 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:04.723191 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:44:04.723233 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":130,"threads_started":1}
I20260806 22:44:04.723483 32487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93643676575643ac8c7959369dbbb9af. Latest consensus state: current_term: 1 leader_uuid: "93643676575643ac8c7959369dbbb9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } } }
I20260806 22:44:04.723554 32487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:04.723766 32512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:04.724246 32495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93643676575643ac8c7959369dbbb9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93643676575643ac8c7959369dbbb9af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38111 } } }
I20260806 22:44:04.724385 32495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:04.724428 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:04.724474 32512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:04.727056 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:04.726217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58999 (local address 127.0.0.1:38111)
0806 22:44:04.726249 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:04.726251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:04.726264 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:04.726336 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:04.726336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:04.726350 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:04.726394 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:04.726846 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:04.726847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:04.726847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:04.726851 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:04.726862 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:04.726865 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:04.726905 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:04.726959 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:44:04.726968 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:44:04.727636 32512 catalog_manager.cc:1347] Generated new cluster ID: fa4ff60ae8db4808a5ca18e29bd41c31
I20260806 22:44:04.727684 32512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:04.759985 32512 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:04.760164 32512 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:04.764717 32512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93643676575643ac8c7959369dbbb9af: Generated new TSK 0
I20260806 22:44:04.764911 32512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:04.766368 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 32389
2026-08-06T22:44:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:04 dist-test-slave-gjrz krb5kdc[32538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:04 dist-test-slave-gjrz krb5kdc[32538](info): set up 2 sockets
Aug 06 22:44:04 dist-test-slave-gjrz krb5kdc[32538](info): commencing operation
krb5kdc: starting...
W20260806 22:44:05.892097 747 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:05 dist-test-slave-gjrz krb5kdc[32538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:06.005722 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:06.087129 32558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:06.087347 32558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:06.087378 32558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:06.088481 32558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:06.088519 32558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:06.088532 32558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:06.088543 32558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:06.088559 32558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:06.089972 32558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:06.090163 32558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:06.090334 32558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:06.094933 32570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:06.094906 32573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:06.094981 32558 server_base.cc:1034] running on GCE node
W20260806 22:44:06.094919 32568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:06.095382 32558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:06.095667 32558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:06.096810 32558 hybrid_clock.cc:648] HybridClock initialized: now 1786056246096796 us; error 26 us; skew 500 ppm
Aug 06 22:44:06 dist-test-slave-gjrz krb5kdc[32538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:06.099167 32558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:06.099632 32558 webserver.cc:460] Webserver started at http://127.0.0.1:37835/ using document root <none> and password file <none>
I20260806 22:44:06.099833 32558 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:06.099881 32558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:06.100005 32558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:06.101023 32558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "b1b2821f540540fea2050f2ba56f8c03"
format_stamp: "Formatted at 2026-08-06 22:44:06 on dist-test-slave-gjrz"
server_key: "29c7b635595c38ce9b1c1cd849b410b0"
server_key_iv: "5b78a128bcfd460ef917b907923ca9a9"
server_key_version: "encryptionkey@0"
I20260806 22:44:06.101356 32558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "b1b2821f540540fea2050f2ba56f8c03"
format_stamp: "Formatted at 2026-08-06 22:44:06 on dist-test-slave-gjrz"
server_key: "29c7b635595c38ce9b1c1cd849b410b0"
server_key_iv: "5b78a128bcfd460ef917b907923ca9a9"
server_key_version: "encryptionkey@0"
I20260806 22:44:06.102571 32558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:06.103188 32581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:06.103348 32558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:06.103440 32558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "b1b2821f540540fea2050f2ba56f8c03"
format_stamp: "Formatted at 2026-08-06 22:44:06 on dist-test-slave-gjrz"
server_key: "29c7b635595c38ce9b1c1cd849b410b0"
server_key_iv: "5b78a128bcfd460ef917b907923ca9a9"
server_key_version: "encryptionkey@0"
I20260806 22:44:06.103509 32558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:06.124359 32558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:06.126196 32558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:06.126333 32558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:06.130229 32558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46125
I20260806 22:44:06.130266 32658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46125 every 8 connection(s)
I20260806 22:44:06.130604 32558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:06.131487 32659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:06.133725 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03: Bootstrap starting.
I20260806 22:44:06.134268 32659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:06.134496 32659 log.cc:824] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:06.135134 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03: No bootstrap required, opened a new log
I20260806 22:44:06.136729 32659 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260806 22:44:06.136864 32659 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:06.136885 32659 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1b2821f540540fea2050f2ba56f8c03, State: Initialized, Role: FOLLOWER
I20260806 22:44:06.136997 32659 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260806 22:44:06.137073 32659 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:06.137102 32659 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:06.137136 32659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:06.137616 32659 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260806 22:44:06.137707 32659 leader_election.cc:302] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1b2821f540540fea2050f2ba56f8c03; no voters:
I20260806 22:44:06.137830 32659 leader_election.cc:288] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:06.137878 32663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:06.138033 32663 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [term 1 LEADER]: Becoming Leader. State: Replica: b1b2821f540540fea2050f2ba56f8c03, State: Running, Role: LEADER
I20260806 22:44:06.138064 32659 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:06.138126 32663 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260806 22:44:06.138478 32668 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1b2821f540540fea2050f2ba56f8c03. Latest consensus state: current_term: 1 leader_uuid: "b1b2821f540540fea2050f2ba56f8c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } } }
I20260806 22:44:06.138595 32664 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1b2821f540540fea2050f2ba56f8c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b2821f540540fea2050f2ba56f8c03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } } }
I20260806 22:44:06.138620 32668 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:06.138684 32664 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:06.139101 32680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:06.139578 32680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:06.140419 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 32558
I20260806 22:44:06.140542 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:06.140712 747 external_mini_cluster.cc:1428] Setting key 03ed9c1f737612e4b13636f2639e3a9a
I20260806 22:44:06.141057 32680 catalog_manager.cc:1347] Generated new cluster ID: 6dedfd7e622e4e4facf6b128a4701928
I20260806 22:44:06.141108 32680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:06 dist-test-slave-gjrz krb5kdc[32538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:44:06.150134 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:06.141359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45381 (local address 127.0.0.1:46125)
0806 22:44:06.141514 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:06.141524 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:06.141548 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:44:06.141637 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:06.141642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:06.141683 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:06.141792 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:06.142418 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:06.143406 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:06.144117 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:06.144529 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:06.144623 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:06.146780 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:06.146784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:06.146784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:06.146796 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:06.148727 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:06.149045 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:06.149048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:06.149048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:06.149146 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:06.149315 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:06.149316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:06.149316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:06.149557 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:06.149699 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:06.149794 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:44:06.149924 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 22:44:06.150967 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:06.153002 32680 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:06.153644 32680 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:06.157763 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:06.152267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36469 (local address 127.0.0.1:46125)
0806 22:44:06.152301 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:06.152303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:06.152427 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:44:06.152482 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:06.152483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:06.152496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:06.152545 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:06.153156 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:06.153629 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:06.154291 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:06.154509 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:06.154561 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:06.155610 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:06.155611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:06.155611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:06.155618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:06.156803 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:06.157105 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:06.157106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:06.157106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:06.157159 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:06.157316 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:06.157316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:06.157317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:06.157421 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:06.157523 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:06.157599 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:44:06.157681 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:44:06.160575 32680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1b2821f540540fea2050f2ba56f8c03: Generated new TSK 0
I20260806 22:44:06.160740 32680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:06.164552 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 32558
2026-08-06T22:44:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:06 dist-test-slave-gjrz krb5kdc[32709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:06 dist-test-slave-gjrz krb5kdc[32709](info): set up 2 sockets
Aug 06 22:44:06 dist-test-slave-gjrz krb5kdc[32709](info): commencing operation
krb5kdc: starting...
W20260806 22:44:07.297102 747 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:07 dist-test-slave-gjrz krb5kdc[32709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:07.411548 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:07.490861 32728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:07.491120 32728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:07.491153 32728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:07.492906 32728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:07.492964 32728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:07.493041 32728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:07.493073 32728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:07.493091 32728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:07.495190 32728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:07.495438 32728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:07.495657 32728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:07.501222 32739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:07.501243 32738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:07.501243 32742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:07.501443 32728 server_base.cc:1034] running on GCE node
I20260806 22:44:07.501677 32728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:07.501957 32728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:07.503134 32728 hybrid_clock.cc:648] HybridClock initialized: now 1786056247503124 us; error 29 us; skew 500 ppm
Aug 06 22:44:07 dist-test-slave-gjrz krb5kdc[32709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056247, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:07.505631 32728 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:07.506004 32728 webserver.cc:460] Webserver started at http://127.0.0.1:45755/ using document root <none> and password file <none>
I20260806 22:44:07.506219 32728 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:07.506261 32728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:07.506349 32728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:07.507359 32728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "dfaea206f0344b2ba905cadb35406e9a"
format_stamp: "Formatted at 2026-08-06 22:44:07 on dist-test-slave-gjrz"
server_key: "26ca6131173312780bee4109891f05e5"
server_key_iv: "5218d88231e02d147e6e93d643c3b8bf"
server_key_version: "encryptionkey@0"
I20260806 22:44:07.507653 32728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "dfaea206f0344b2ba905cadb35406e9a"
format_stamp: "Formatted at 2026-08-06 22:44:07 on dist-test-slave-gjrz"
server_key: "26ca6131173312780bee4109891f05e5"
server_key_iv: "5218d88231e02d147e6e93d643c3b8bf"
server_key_version: "encryptionkey@0"
I20260806 22:44:07.508806 32728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:07.509454 32752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:07.509600 32728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:07.509681 32728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "dfaea206f0344b2ba905cadb35406e9a"
format_stamp: "Formatted at 2026-08-06 22:44:07 on dist-test-slave-gjrz"
server_key: "26ca6131173312780bee4109891f05e5"
server_key_iv: "5218d88231e02d147e6e93d643c3b8bf"
server_key_version: "encryptionkey@0"
I20260806 22:44:07.509742 32728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:07.531006 32728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:07.532925 32728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:07.533057 32728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:07.536878 32728 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46373
I20260806 22:44:07.536923 364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46373 every 8 connection(s)
I20260806 22:44:07.537236 32728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:07.538113 366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:07.540479 366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: Bootstrap starting.
I20260806 22:44:07.541043 366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:07.541312 366 log.cc:824] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:07.542002 366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: No bootstrap required, opened a new log
I20260806 22:44:07.543658 366 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } }
I20260806 22:44:07.543794 366 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:07.543828 366 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfaea206f0344b2ba905cadb35406e9a, State: Initialized, Role: FOLLOWER
I20260806 22:44:07.543965 366 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } }
I20260806 22:44:07.544031 366 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:07.544064 366 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:07.544112 366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:07.544623 366 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } }
I20260806 22:44:07.544718 366 leader_election.cc:302] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfaea206f0344b2ba905cadb35406e9a; no voters:
I20260806 22:44:07.544884 366 leader_election.cc:288] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:07.544914 370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:07.545061 370 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [term 1 LEADER]: Becoming Leader. State: Replica: dfaea206f0344b2ba905cadb35406e9a, State: Running, Role: LEADER
I20260806 22:44:07.545121 366 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:07.545176 370 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } }
I20260806 22:44:07.545740 371 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfaea206f0344b2ba905cadb35406e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } } }
I20260806 22:44:07.545892 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 32728
I20260806 22:44:07.546015 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:07.546007 371 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:07.545933 372 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfaea206f0344b2ba905cadb35406e9a. Latest consensus state: current_term: 1 leader_uuid: "dfaea206f0344b2ba905cadb35406e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaea206f0344b2ba905cadb35406e9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46373 } } }
I20260806 22:44:07.546159 372 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:07.546201 747 external_mini_cluster.cc:1428] Setting key 0ce04b1b3d19385221c46b23a3352fcf
W20260806 22:44:07.546892 393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:07.546952 393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:07.547003 386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:07.547175 386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:07.548585 386 catalog_manager.cc:1347] Generated new cluster ID: 46685630fa024c95866385b7aeedbbbc
I20260806 22:44:07.548636 386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:07.552280 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:07.546857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60771 (local address 127.0.0.1:46373)
0806 22:44:07.547058 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:07.547067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:07.547100 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:44:07.547213 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:07.547220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:07.547293 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:07.547426 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:07.547662 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:07.548994 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:07.550054 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:07.550666 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:07.550778 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:07.551485 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:07.551490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:07.551490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:07.551708 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:07.551746 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:07.551761 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:07.551831 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:07.551891 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:44:07.552066 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260806 22:44:07.553290 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:07.558398 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:07.555014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39417 (local address 127.0.0.1:46373)
0806 22:44:07.555056 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:07.555058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:07.555207 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:44:07.555289 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:07.555289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:07.555299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:07.555346 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:07.555497 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:07.556190 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:07.557198 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:07.557495 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:07.557552 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:07.558178 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:07.558179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:07.558180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:07.558184 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:07.558199 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:07.558204 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:07.558246 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:07.558301 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:44:07.558325 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:07.572858 386 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:07.573549 386 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:07.578022 386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfaea206f0344b2ba905cadb35406e9a: Generated new TSK 0
I20260806 22:44:07.578192 386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:07.586593 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 32728
2026-08-06T22:44:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:07 dist-test-slave-gjrz krb5kdc[413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:07 dist-test-slave-gjrz krb5kdc[413](info): set up 2 sockets
Aug 06 22:44:07 dist-test-slave-gjrz krb5kdc[413](info): commencing operation
krb5kdc: starting...
W20260806 22:44:08.710064 747 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:08 dist-test-slave-gjrz krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:08.821552 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:08.902259 431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:08.902462 431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:08.902491 431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:08.903676 431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:08.903719 431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:08.903740 431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:08.903754 431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:08.903771 431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:08.905263 431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:08.905510 431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:08.905721 431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:08.910295 443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:08.910338 441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:08.910301 446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:08.910604 431 server_base.cc:1034] running on GCE node
I20260806 22:44:08.910830 431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:08.911103 431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:08.912246 431 hybrid_clock.cc:648] HybridClock initialized: now 1786056248912219 us; error 42 us; skew 500 ppm
Aug 06 22:44:08 dist-test-slave-gjrz krb5kdc[413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:08.914610 431 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:08.915045 431 webserver.cc:460] Webserver started at http://127.0.0.1:44031/ using document root <none> and password file <none>
I20260806 22:44:08.915215 431 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:08.915253 431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:08.915383 431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:08.916345 431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "a284c16161fc4e7087b05698d92c181c"
format_stamp: "Formatted at 2026-08-06 22:44:08 on dist-test-slave-gjrz"
server_key: "002bc9db6e56156f70a4ce2be4aaf734"
server_key_iv: "a9920e5ade2f0f4f18426d9fd6a85a90"
server_key_version: "encryptionkey@0"
I20260806 22:44:08.916647 431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "a284c16161fc4e7087b05698d92c181c"
format_stamp: "Formatted at 2026-08-06 22:44:08 on dist-test-slave-gjrz"
server_key: "002bc9db6e56156f70a4ce2be4aaf734"
server_key_iv: "a9920e5ade2f0f4f18426d9fd6a85a90"
server_key_version: "encryptionkey@0"
I20260806 22:44:08.917786 431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:08.918423 453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:08.918565 431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:08.918659 431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "a284c16161fc4e7087b05698d92c181c"
format_stamp: "Formatted at 2026-08-06 22:44:08 on dist-test-slave-gjrz"
server_key: "002bc9db6e56156f70a4ce2be4aaf734"
server_key_iv: "a9920e5ade2f0f4f18426d9fd6a85a90"
server_key_version: "encryptionkey@0"
I20260806 22:44:08.918715 431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:08.921785 431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:08.923418 431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:08.923521 431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:08.927183 431 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39131
I20260806 22:44:08.927227 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39131 every 8 connection(s)
I20260806 22:44:08.927510 431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:08.928360 534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:08.930452 534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c: Bootstrap starting.
I20260806 22:44:08.930969 534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:08.931221 534 log.cc:824] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:08.931902 534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c: No bootstrap required, opened a new log
I20260806 22:44:08.933442 534 raft_consensus.cc:348] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } }
I20260806 22:44:08.933571 534 raft_consensus.cc:374] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:08.933601 534 raft_consensus.cc:729] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a284c16161fc4e7087b05698d92c181c, State: Initialized, Role: FOLLOWER
I20260806 22:44:08.933697 534 consensus_queue.cc:260] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } }
I20260806 22:44:08.933745 534 raft_consensus.cc:388] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:08.933769 534 raft_consensus.cc:482] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:08.933799 534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:08.934278 534 raft_consensus.cc:504] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } }
I20260806 22:44:08.934360 534 leader_election.cc:302] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a284c16161fc4e7087b05698d92c181c; no voters:
I20260806 22:44:08.934484 534 leader_election.cc:288] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:08.934527 539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:08.934689 539 raft_consensus.cc:686] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [term 1 LEADER]: Becoming Leader. State: Replica: a284c16161fc4e7087b05698d92c181c, State: Running, Role: LEADER
I20260806 22:44:08.934707 534 sys_catalog.cc:564] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:08.934773 539 consensus_queue.cc:237] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } }
I20260806 22:44:08.935122 540 sys_catalog.cc:455] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a284c16161fc4e7087b05698d92c181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } } }
I20260806 22:44:08.935231 540 sys_catalog.cc:458] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:08.935114 543 sys_catalog.cc:455] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a284c16161fc4e7087b05698d92c181c. Latest consensus state: current_term: 1 leader_uuid: "a284c16161fc4e7087b05698d92c181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a284c16161fc4e7087b05698d92c181c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39131 } } }
I20260806 22:44:08.935365 543 sys_catalog.cc:458] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:08.935671 545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:08.936012 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 431
I20260806 22:44:08.936125 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:08.936244 545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:08.936307 747 external_mini_cluster.cc:1428] Setting key 2a01e3f1447c3f455a8ee401ce80dd1e
I20260806 22:44:08.937675 545 catalog_manager.cc:1347] Generated new cluster ID: 8b43449a81104d7a8c64821fe993be8b
I20260806 22:44:08.937722 545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:08.938670 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:08.936777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52205 (local address 127.0.0.1:39131)
0806 22:44:08.937163 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:08.937171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:08.937202 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:44:08.937322 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:08.937329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:08.937357 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:08.937488 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:08.938267 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:08.938271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:08.938271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:08.938276 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:08.938317 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:08.938331 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:08.938403 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:08.938457 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:44:08.938498 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":79,"threads_started":1}
I20260806 22:44:08.939522 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:08.941917 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:08.940896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52391 (local address 127.0.0.1:39131)
0806 22:44:08.940994 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:08.940996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:08.941118 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:44:08.941179 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:08.941179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:08.941183 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:08.941234 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:08.941733 (+ 499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:08.941734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:08.941735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:08.941738 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:08.941751 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:08.941754 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:08.941795 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:08.941839 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:44:08.941848 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:44:08.950436 545 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:08.950595 545 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:08.958786 545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a284c16161fc4e7087b05698d92c181c: Generated new TSK 0
I20260806 22:44:08.958942 545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:08.961592 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 431
2026-08-06T22:44:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:09 dist-test-slave-gjrz krb5kdc[579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:09 dist-test-slave-gjrz krb5kdc[579](info): set up 2 sockets
Aug 06 22:44:09 dist-test-slave-gjrz krb5kdc[579](info): commencing operation
krb5kdc: starting...
W20260806 22:44:10.094179 747 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:10.207229 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:10.287380 600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:10.287593 600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:10.287627 600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:10.288800 600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:10.288846 600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:10.288866 600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:10.288882 600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:10.288897 600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:10.290400 600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:10.290653 600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:10.290865 600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:10.295475 609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:10.295504 608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:10.295475 612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:10.295748 600 server_base.cc:1034] running on GCE node
I20260806 22:44:10.295975 600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:10.296253 600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:10.297415 600 hybrid_clock.cc:648] HybridClock initialized: now 1786056250297393 us; error 32 us; skew 500 ppm
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056250, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:10.302963 600 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:10.303452 600 webserver.cc:460] Webserver started at http://127.0.0.1:41929/ using document root <none> and password file <none>
I20260806 22:44:10.303666 600 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:10.303706 600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:10.303798 600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:10.304797 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "9a34adbed9c94343bc4f1c41abd692bd"
format_stamp: "Formatted at 2026-08-06 22:44:10 on dist-test-slave-gjrz"
server_key: "d44add92cd6f8542fd00eb50b07c3b53"
server_key_iv: "2fac69313b1752e26d035e6b05288ba3"
server_key_version: "encryptionkey@0"
I20260806 22:44:10.305115 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "9a34adbed9c94343bc4f1c41abd692bd"
format_stamp: "Formatted at 2026-08-06 22:44:10 on dist-test-slave-gjrz"
server_key: "d44add92cd6f8542fd00eb50b07c3b53"
server_key_iv: "2fac69313b1752e26d035e6b05288ba3"
server_key_version: "encryptionkey@0"
I20260806 22:44:10.306324 600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:10.306972 621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:10.307152 600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:10.307221 600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "9a34adbed9c94343bc4f1c41abd692bd"
format_stamp: "Formatted at 2026-08-06 22:44:10 on dist-test-slave-gjrz"
server_key: "d44add92cd6f8542fd00eb50b07c3b53"
server_key_iv: "2fac69313b1752e26d035e6b05288ba3"
server_key_version: "encryptionkey@0"
I20260806 22:44:10.307317 600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:10.335830 600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:10.337723 600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:10.337847 600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:10.341923 600 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41095
I20260806 22:44:10.341979 713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41095 every 8 connection(s)
I20260806 22:44:10.342322 600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:10.343190 714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:10.345669 714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd: Bootstrap starting.
I20260806 22:44:10.346228 714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:10.346494 714 log.cc:824] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:10.347196 714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd: No bootstrap required, opened a new log
I20260806 22:44:10.348838 714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } }
I20260806 22:44:10.348976 714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:10.348997 714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a34adbed9c94343bc4f1c41abd692bd, State: Initialized, Role: FOLLOWER
I20260806 22:44:10.349133 714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } }
I20260806 22:44:10.349207 714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:10.349232 714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:10.349279 714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:10.349802 714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } }
I20260806 22:44:10.349895 714 leader_election.cc:302] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a34adbed9c94343bc4f1c41abd692bd; no voters:
I20260806 22:44:10.350057 714 leader_election.cc:288] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:10.350096 719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:10.350248 719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [term 1 LEADER]: Becoming Leader. State: Replica: 9a34adbed9c94343bc4f1c41abd692bd, State: Running, Role: LEADER
I20260806 22:44:10.350304 714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:10.350342 719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } }
I20260806 22:44:10.350709 721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a34adbed9c94343bc4f1c41abd692bd. Latest consensus state: current_term: 1 leader_uuid: "9a34adbed9c94343bc4f1c41abd692bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } } }
I20260806 22:44:10.350788 721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:10.350838 720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a34adbed9c94343bc4f1c41abd692bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a34adbed9c94343bc4f1c41abd692bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41095 } } }
I20260806 22:44:10.350916 720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:10.351248 728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:10.351763 728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:10.352020 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 600
I20260806 22:44:10.352141 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:10.352363 747 external_mini_cluster.cc:1428] Setting key fe60f7b8e745af68d72ac17a9a561179
I20260806 22:44:10.353713 728 catalog_manager.cc:1347] Generated new cluster ID: d219e7f08fb249b5b39b4bd02190a31e
I20260806 22:44:10.353760 728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:44:10.363488 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:10.352887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:35461 (local address 127.0.0.1:41095)
0806 22:44:10.353078 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:10.353086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:10.353112 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:44:10.353202 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:10.353208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:10.353251 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:10.353357 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:10.354128 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:10.355506 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:10.356539 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:10.357125 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:10.357212 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:10.359975 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:10.359979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:10.359979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:10.359992 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:10.361824 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:10.362281 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:10.362283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:10.362284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:10.362381 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:10.362587 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:10.362587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:10.362588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:10.362857 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:10.363016 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:10.363136 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:44:10.363228 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260806 22:44:10.364454 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:10.368350 728 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:10.368955 728 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:10.372430 728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a34adbed9c94343bc4f1c41abd692bd: Generated new TSK 0
I20260806 22:44:10.372591 728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:10.373010 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:10.365869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:37345 (local address 127.0.0.1:41095)
0806 22:44:10.365905 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:10.365907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:10.366026 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:44:10.366096 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:10.366097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:10.366106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:10.366148 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:10.366718 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:10.367634 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:10.368640 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:10.368940 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:10.369008 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:10.370308 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:10.370309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:10.370309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:10.370315 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:10.371733 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:10.372154 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:10.372155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:10.372155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:10.372215 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:10.372458 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:10.372459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:10.372459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:10.372576 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:10.372656 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:10.372873 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:44:10.372928 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":16}
I20260806 22:44:10.374872 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 600
W20260806 22:44:10.386175 577 connection.cc:535] client connection to 127.0.0.1:41095 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41095): Connection reset by peer (error 104)
2026-08-06T22:44:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[768](info): set up 2 sockets
Aug 06 22:44:10 dist-test-slave-gjrz krb5kdc[768](info): commencing operation
krb5kdc: starting...
W20260806 22:44:11.497365 747 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:11 dist-test-slave-gjrz krb5kdc[768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:11.611222 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:11.691453 793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:11.691645 793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:11.691676 793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:11.692791 793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:11.692830 793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:11.692850 793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:11.692865 793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:11.692881 793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:11.694337 793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:11.694574 793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:11.694772 793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:11.700809 800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:11.700965 793 server_base.cc:1034] running on GCE node
W20260806 22:44:11.700815 803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:11.700815 801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:11.701292 793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:11.701584 793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:11.702724 793 hybrid_clock.cc:648] HybridClock initialized: now 1786056251702710 us; error 28 us; skew 500 ppm
Aug 06 22:44:11 dist-test-slave-gjrz krb5kdc[768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:11.705078 793 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:11.705482 793 webserver.cc:460] Webserver started at http://127.0.0.1:42013/ using document root <none> and password file <none>
I20260806 22:44:11.705684 793 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:11.705724 793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:11.705812 793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:11.706763 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "037cd82793604262aea544240c5ec499"
format_stamp: "Formatted at 2026-08-06 22:44:11 on dist-test-slave-gjrz"
server_key: "e7e1a84eb5a85c15a2691d742d07b7c8"
server_key_iv: "53c41be1d80ab4990b8e4634fd7b37d7"
server_key_version: "encryptionkey@0"
I20260806 22:44:11.707060 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "037cd82793604262aea544240c5ec499"
format_stamp: "Formatted at 2026-08-06 22:44:11 on dist-test-slave-gjrz"
server_key: "e7e1a84eb5a85c15a2691d742d07b7c8"
server_key_iv: "53c41be1d80ab4990b8e4634fd7b37d7"
server_key_version: "encryptionkey@0"
I20260806 22:44:11.708300 793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:44:11.708943 812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:11.709113 793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:11.709195 793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "037cd82793604262aea544240c5ec499"
format_stamp: "Formatted at 2026-08-06 22:44:11 on dist-test-slave-gjrz"
server_key: "e7e1a84eb5a85c15a2691d742d07b7c8"
server_key_iv: "53c41be1d80ab4990b8e4634fd7b37d7"
server_key_version: "encryptionkey@0"
I20260806 22:44:11.709249 793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:11.718593 793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:11.720187 793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:11.720288 793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:11.723940 793 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41761
I20260806 22:44:11.723977 898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41761 every 8 connection(s)
I20260806 22:44:11.724277 793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:11.725100 899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:11.727217 899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499: Bootstrap starting.
I20260806 22:44:11.727627 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 793
I20260806 22:44:11.727730 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:11.727830 899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:11.727900 747 external_mini_cluster.cc:1428] Setting key cdcb82649f82763f8843375e072d9de2
I20260806 22:44:11.728164 899 log.cc:824] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:11.728894 899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499: No bootstrap required, opened a new log
I20260806 22:44:11.731240 899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260806 22:44:11.731410 899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:11.731436 899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037cd82793604262aea544240c5ec499, State: Initialized, Role: FOLLOWER
I20260806 22:44:11.731541 899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260806 22:44:11.731607 899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:11.731628 899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:11.731670 899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:11.732188 899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260806 22:44:11.732288 899 leader_election.cc:302] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 037cd82793604262aea544240c5ec499; no voters:
I20260806 22:44:11.732442 899 leader_election.cc:288] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:11.732482 907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:11.732668 907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [term 1 LEADER]: Becoming Leader. State: Replica: 037cd82793604262aea544240c5ec499, State: Running, Role: LEADER
I20260806 22:44:11.732704 899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:11.732815 907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } }
I20260806 22:44:11.733443 910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "037cd82793604262aea544240c5ec499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } } }
I20260806 22:44:11.733593 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:11.728371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52651 (local address 127.0.0.1:41761)
0806 22:44:11.728531 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:11.728541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:11.728581 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:44:11.729121 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:11.729130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:11.729191 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:11.729336 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:11.729508 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:11.730863 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:11.731699 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:11.732127 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:11.732224 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:11.732885 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:11.732889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:11.732889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:11.733070 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:11.733108 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:11.733122 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:11.733195 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:11.733345 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:44:11.733368 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260806 22:44:11.733596 910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:11.733848 908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 037cd82793604262aea544240c5ec499. Latest consensus state: current_term: 1 leader_uuid: "037cd82793604262aea544240c5ec499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037cd82793604262aea544240c5ec499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41761 } } }
I20260806 22:44:11.733934 908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:11.734632 928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:11.734736 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:11.735092 928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:11.736567 928 catalog_manager.cc:1347] Generated new cluster ID: eca8663e9211425b9ca37e202bb30d81
I20260806 22:44:11.736617 928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:11.745658 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:11.736027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:59351 (local address 127.0.0.1:41761)
0806 22:44:11.736059 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:11.736061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:11.736274 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:44:11.736331 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:11.736331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:11.736340 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:11.736396 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:11.743438 (+ 7042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:11.743917 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:11.744599 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:11.744807 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:11.744860 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:11.745423 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:11.745424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:11.745424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:11.745429 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:11.745442 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:11.745447 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:11.745489 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:11.745566 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:44:11.745581 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:44:11.763936 928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:11.764429 928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:11.773123 928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 037cd82793604262aea544240c5ec499: Generated new TSK 0
I20260806 22:44:11.773319 928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:11.783237 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 793
W20260806 22:44:11.796361 766 connection.cc:535] client connection to 127.0.0.1:41761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41761): Connection reset by peer (error 104)
2026-08-06T22:44:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:11 dist-test-slave-gjrz krb5kdc[942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:11 dist-test-slave-gjrz krb5kdc[942](info): set up 2 sockets
Aug 06 22:44:11 dist-test-slave-gjrz krb5kdc[942](info): commencing operation
krb5kdc: starting...
W20260806 22:44:12.911514 747 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:12 dist-test-slave-gjrz krb5kdc[942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:13.027576 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:13.109814 963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:13.110028 963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:13.110060 963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:13.111215 963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:13.111284 963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:13.111306 963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:13.111323 963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:13.111347 963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:13.112872 963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:13.113133 963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:13.113351 963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:13.118120 973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:13.118134 969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:13.118132 971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:13.118407 963 server_base.cc:1034] running on GCE node
I20260806 22:44:13.118634 963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:13.118894 963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:13.120026 963 hybrid_clock.cc:648] HybridClock initialized: now 1786056253120015 us; error 31 us; skew 500 ppm
Aug 06 22:44:13 dist-test-slave-gjrz krb5kdc[942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:13.122390 963 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:13.122792 963 webserver.cc:460] Webserver started at http://127.0.0.1:42611/ using document root <none> and password file <none>
I20260806 22:44:13.122992 963 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:13.123040 963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:13.123171 963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:13.124249 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "45a564c2c7294c6f93a4ba6b19e1946e"
format_stamp: "Formatted at 2026-08-06 22:44:13 on dist-test-slave-gjrz"
server_key: "c75a63f20911d0fb3c5ac71bd09f3f74"
server_key_iv: "e7344e815beadd33bbd3c432b9f2c27b"
server_key_version: "encryptionkey@0"
I20260806 22:44:13.124588 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "45a564c2c7294c6f93a4ba6b19e1946e"
format_stamp: "Formatted at 2026-08-06 22:44:13 on dist-test-slave-gjrz"
server_key: "c75a63f20911d0fb3c5ac71bd09f3f74"
server_key_iv: "e7344e815beadd33bbd3c432b9f2c27b"
server_key_version: "encryptionkey@0"
I20260806 22:44:13.125813 963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:44:13.126473 985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:13.126619 963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:13.126703 963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "45a564c2c7294c6f93a4ba6b19e1946e"
format_stamp: "Formatted at 2026-08-06 22:44:13 on dist-test-slave-gjrz"
server_key: "c75a63f20911d0fb3c5ac71bd09f3f74"
server_key_iv: "e7344e815beadd33bbd3c432b9f2c27b"
server_key_version: "encryptionkey@0"
I20260806 22:44:13.126762 963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:13.129956 963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:13.131695 963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:13.131815 963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:13.135504 963 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36905
I20260806 22:44:13.135537 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36905 every 8 connection(s)
I20260806 22:44:13.135823 963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:13.136660 1064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:13.138831 1064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e: Bootstrap starting.
I20260806 22:44:13.139405 1064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:13.139680 1064 log.cc:824] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:13.140291 1064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e: No bootstrap required, opened a new log
I20260806 22:44:13.141861 1064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260806 22:44:13.142005 1064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:13.142037 1064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a564c2c7294c6f93a4ba6b19e1946e, State: Initialized, Role: FOLLOWER
I20260806 22:44:13.142172 1064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260806 22:44:13.142256 1064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:13.142288 1064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:13.142338 1064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:13.142328 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 963
I20260806 22:44:13.142441 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:13.142619 747 external_mini_cluster.cc:1428] Setting key ed7049d8233bfad11670ed31fab5155e
I20260806 22:44:13.142935 1064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260806 22:44:13.143057 1064 leader_election.cc:302] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45a564c2c7294c6f93a4ba6b19e1946e; no voters:
I20260806 22:44:13.143220 1064 leader_election.cc:288] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:13.143246 1074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:13.143456 1074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [term 1 LEADER]: Becoming Leader. State: Replica: 45a564c2c7294c6f93a4ba6b19e1946e, State: Running, Role: LEADER
I20260806 22:44:13.143551 1064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:13.143582 1074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260806 22:44:13.143895 1074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45a564c2c7294c6f93a4ba6b19e1946e. Latest consensus state: current_term: 1 leader_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260806 22:44:13.144102 1074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:13.144030 1075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a564c2c7294c6f93a4ba6b19e1946e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260806 22:44:13.144538 1075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:13.144652 1080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:13.145143 1080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:13.145179 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:13.143031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:60689 (local address 127.0.0.1:36905)
0806 22:44:13.143222 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:13.143231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:13.143290 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:44:13.143437 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:13.143445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:13.143475 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:13.144364 (+ 889us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:13.144673 (+ 309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:13.144677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:13.144677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:13.144680 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:13.144713 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:13.144727 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:13.144792 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:13.144957 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:44:13.144999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260806 22:44:13.146492 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:13.146739 1080 catalog_manager.cc:1347] Generated new cluster ID: a98a6d959546488e9347247ebd3862d2
I20260806 22:44:13.146782 1080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:13.152379 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:13.148014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:34133 (local address 127.0.0.1:36905)
0806 22:44:13.148044 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:13.148046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:13.151468 (+ 3422us) server_negotiation.cc:407] Connection header received
0806 22:44:13.151541 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:13.151541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:13.151547 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:13.151617 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:13.152175 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:13.152176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:13.152177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:13.152181 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:13.152196 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:13.152200 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:13.152241 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:13.152290 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:44:13.152302 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:44:13.173023 1080 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:13.173218 1080 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:13.183012 1080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45a564c2c7294c6f93a4ba6b19e1946e: Generated new TSK 0
I20260806 22:44:13.183208 1080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:13.185468 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 963
2026-08-06T22:44:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:13 dist-test-slave-gjrz krb5kdc[1111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:13 dist-test-slave-gjrz krb5kdc[1111](info): set up 2 sockets
Aug 06 22:44:13 dist-test-slave-gjrz krb5kdc[1111](info): commencing operation
krb5kdc: starting...
W20260806 22:44:14.314056 747 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:14.426034 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:14.505445 1127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:14.505640 1127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:14.505669 1127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:14.506814 1127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:14.506862 1127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:14.506880 1127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:14.506896 1127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:14.506910 1127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:14.508399 1127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:14.508649 1127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:14.508846 1127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:14.513307 1138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:14.513475 1127 server_base.cc:1034] running on GCE node
W20260806 22:44:14.513298 1136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:14.513612 1140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:14.513824 1127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:14.514070 1127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:14.515232 1127 hybrid_clock.cc:648] HybridClock initialized: now 1786056254515212 us; error 31 us; skew 500 ppm
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056254, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:14.517562 1127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:14.517956 1127 webserver.cc:460] Webserver started at http://127.0.0.1:43907/ using document root <none> and password file <none>
I20260806 22:44:14.518152 1127 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:14.518198 1127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:14.518322 1127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:14.519359 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "69c16d0f8f42434d8eb3d9657e307d02"
format_stamp: "Formatted at 2026-08-06 22:44:14 on dist-test-slave-gjrz"
server_key: "42baf3ce53eebcb44813fa9155a9a0a1"
server_key_iv: "13736fe1a7900949698cf396ca01bf47"
server_key_version: "encryptionkey@0"
I20260806 22:44:14.519698 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "69c16d0f8f42434d8eb3d9657e307d02"
format_stamp: "Formatted at 2026-08-06 22:44:14 on dist-test-slave-gjrz"
server_key: "42baf3ce53eebcb44813fa9155a9a0a1"
server_key_iv: "13736fe1a7900949698cf396ca01bf47"
server_key_version: "encryptionkey@0"
I20260806 22:44:14.520888 1127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:14.521552 1151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:14.521710 1127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:14.521796 1127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "69c16d0f8f42434d8eb3d9657e307d02"
format_stamp: "Formatted at 2026-08-06 22:44:14 on dist-test-slave-gjrz"
server_key: "42baf3ce53eebcb44813fa9155a9a0a1"
server_key_iv: "13736fe1a7900949698cf396ca01bf47"
server_key_version: "encryptionkey@0"
I20260806 22:44:14.521855 1127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:14.546119 1127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:14.548087 1127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:14.548243 1127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:14.552120 1127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44279
I20260806 22:44:14.552152 1232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44279 every 8 connection(s)
I20260806 22:44:14.552482 1127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:14.553323 1233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:14.555564 1233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02: Bootstrap starting.
I20260806 22:44:14.556110 1233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:14.556336 1233 log.cc:824] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:14.556937 1233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02: No bootstrap required, opened a new log
I20260806 22:44:14.558482 1233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } }
I20260806 22:44:14.558620 1233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:14.558641 1233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c16d0f8f42434d8eb3d9657e307d02, State: Initialized, Role: FOLLOWER
I20260806 22:44:14.558739 1233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } }
I20260806 22:44:14.558794 1233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:14.558821 1233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:14.558861 1233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:14.559415 1233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } }
I20260806 22:44:14.559513 1233 leader_election.cc:302] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69c16d0f8f42434d8eb3d9657e307d02; no voters:
I20260806 22:44:14.559674 1233 leader_election.cc:288] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:14.559703 1236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:14.559865 1236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [term 1 LEADER]: Becoming Leader. State: Replica: 69c16d0f8f42434d8eb3d9657e307d02, State: Running, Role: LEADER
I20260806 22:44:14.559922 1233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:14.559958 1236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } }
I20260806 22:44:14.560492 1237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69c16d0f8f42434d8eb3d9657e307d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } } }
I20260806 22:44:14.560592 1237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:14.560597 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1127
I20260806 22:44:14.560711 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:14.560715 1239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69c16d0f8f42434d8eb3d9657e307d02. Latest consensus state: current_term: 1 leader_uuid: "69c16d0f8f42434d8eb3d9657e307d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c16d0f8f42434d8eb3d9657e307d02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44279 } } }
I20260806 22:44:14.560786 1239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:14.560892 747 external_mini_cluster.cc:1428] Setting key 6890d9e479c4969e6239d0bb7f838a8b
I20260806 22:44:14.560942 1244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:14.561430 1244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:14.563350 1244 catalog_manager.cc:1347] Generated new cluster ID: 3b05c1ea99454ae9926ce897757eb73f
I20260806 22:44:14.563403 1244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:44:14.572229 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:14.561498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:54775 (local address 127.0.0.1:44279)
0806 22:44:14.561666 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:14.561675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:14.561705 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:44:14.561817 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:14.561824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:14.561898 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:14.562738 (+ 840us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:14.562835 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:14.564153 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:14.565220 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:14.565829 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:14.565920 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:14.568664 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:14.568668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:14.568669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:14.568682 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:14.570565 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:14.570998 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:14.571001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:14.571001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:14.571124 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:14.571338 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:14.571339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:14.571339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:14.571655 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:14.571830 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:14.571909 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:44:14.572012 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260806 22:44:14.573283 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:14.575389 1244 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:14.575947 1244 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:14.578197 1244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69c16d0f8f42434d8eb3d9657e307d02: Generated new TSK 0
I20260806 22:44:14.578366 1244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:14.581516 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:14.574741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:47887 (local address 127.0.0.1:44279)
0806 22:44:14.574792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:14.574794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:14.574908 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:44:14.575003 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:14.575004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:14.575015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:14.575077 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:14.575704 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:14.576464 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:14.577416 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:14.577752 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:14.577818 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:14.579148 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:14.579149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:14.579150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:14.579156 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:14.580503 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:14.580833 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:14.580834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:14.580834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:14.580889 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:14.581081 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:14.581082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:14.581082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:14.581201 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:14.581298 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:14.581370 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:44:14.581431 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:14.583966 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1127
W20260806 22:44:14.598515 1108 connection.cc:535] client connection to 127.0.0.1:44279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44279): Connection reset by peer (error 104)
2026-08-06T22:44:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1286](info): set up 2 sockets
Aug 06 22:44:14 dist-test-slave-gjrz krb5kdc[1286](info): commencing operation
krb5kdc: starting...
W20260806 22:44:15.722123 747 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:15 dist-test-slave-gjrz krb5kdc[1286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:15.833590 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:15.913272 1308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:15.913480 1308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:15.913514 1308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:15.914716 1308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:15.914759 1308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:15.914778 1308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:15.914795 1308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:15.914808 1308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:15.916329 1308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:15.916574 1308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:15.916774 1308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:15.921396 1317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:15.921404 1318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:15.921485 1308 server_base.cc:1034] running on GCE node
W20260806 22:44:15.921396 1320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:15.921895 1308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:15.922158 1308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:15.923293 1308 hybrid_clock.cc:648] HybridClock initialized: now 1786056255923281 us; error 27 us; skew 500 ppm
Aug 06 22:44:15 dist-test-slave-gjrz krb5kdc[1286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056255, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:15.931298 1308 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:15.931699 1308 webserver.cc:460] Webserver started at http://127.0.0.1:36269/ using document root <none> and password file <none>
I20260806 22:44:15.931888 1308 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:15.931931 1308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:15.932061 1308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:15.933061 1308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "814d2dc466f74775bcc8bf5f8e835d13"
format_stamp: "Formatted at 2026-08-06 22:44:15 on dist-test-slave-gjrz"
server_key: "4f7de73adabb7641e6c3ce08733596d1"
server_key_iv: "d11ea5bd709d9b1b7f5764f60a519d11"
server_key_version: "encryptionkey@0"
I20260806 22:44:15.933403 1308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "814d2dc466f74775bcc8bf5f8e835d13"
format_stamp: "Formatted at 2026-08-06 22:44:15 on dist-test-slave-gjrz"
server_key: "4f7de73adabb7641e6c3ce08733596d1"
server_key_iv: "d11ea5bd709d9b1b7f5764f60a519d11"
server_key_version: "encryptionkey@0"
I20260806 22:44:15.934613 1308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:15.935290 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:15.935436 1308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:15.935521 1308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "814d2dc466f74775bcc8bf5f8e835d13"
format_stamp: "Formatted at 2026-08-06 22:44:15 on dist-test-slave-gjrz"
server_key: "4f7de73adabb7641e6c3ce08733596d1"
server_key_iv: "d11ea5bd709d9b1b7f5764f60a519d11"
server_key_version: "encryptionkey@0"
I20260806 22:44:15.935580 1308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:15.955119 1308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:15.956894 1308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:15.957001 1308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:15.960716 1308 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36883
I20260806 22:44:15.960755 1421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36883 every 8 connection(s)
I20260806 22:44:15.961056 1308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:15.962087 1422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:15.964462 1422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13: Bootstrap starting.
I20260806 22:44:15.965011 1422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:15.965242 1422 log.cc:824] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:15.965885 1422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13: No bootstrap required, opened a new log
I20260806 22:44:15.967530 1422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } }
I20260806 22:44:15.967669 1422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:15.967689 1422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814d2dc466f74775bcc8bf5f8e835d13, State: Initialized, Role: FOLLOWER
I20260806 22:44:15.967799 1422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } }
I20260806 22:44:15.967877 1422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:15.967932 1422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:15.967985 1422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:15.968081 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1308
I20260806 22:44:15.968202 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:15.968410 747 external_mini_cluster.cc:1428] Setting key 6557cd10f0915c6bcce9e422591fbcfb
I20260806 22:44:15.968664 1422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } }
I20260806 22:44:15.968767 1422 leader_election.cc:302] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 814d2dc466f74775bcc8bf5f8e835d13; no voters:
I20260806 22:44:15.968991 1422 leader_election.cc:288] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:15.969024 1429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:15.969244 1429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [term 1 LEADER]: Becoming Leader. State: Replica: 814d2dc466f74775bcc8bf5f8e835d13, State: Running, Role: LEADER
I20260806 22:44:15.969328 1422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:15.969360 1429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } }
I20260806 22:44:15.969866 1431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "814d2dc466f74775bcc8bf5f8e835d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } } }
I20260806 22:44:15.969971 1431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:15.970372 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 814d2dc466f74775bcc8bf5f8e835d13. Latest consensus state: current_term: 1 leader_uuid: "814d2dc466f74775bcc8bf5f8e835d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814d2dc466f74775bcc8bf5f8e835d13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36883 } } }
I20260806 22:44:15.970525 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:15.970695 1439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:15.971299 1439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:15.973819 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:15.968879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:37983 (local address 127.0.0.1:36883)
0806 22:44:15.969056 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:15.969065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:15.969130 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:44:15.969255 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:15.969262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:15.969325 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:15.969464 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:15.969703 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:15.970994 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:15.972112 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:15.972524 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:15.972610 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:15.973263 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:15.973266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:15.973267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:15.973421 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:15.973454 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:15.973465 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:15.973529 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:15.973644 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:44:15.973667 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260806 22:44:15.974840 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:15.975108 1439 catalog_manager.cc:1347] Generated new cluster ID: 4984aee23a9242a1b417a5ee3060192e
I20260806 22:44:15.975159 1439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:15.982556 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:15.976326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:50685 (local address 127.0.0.1:36883)
0806 22:44:15.979347 (+ 3021us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:15.979348 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:15.979356 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:15.979375 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:15.979375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:15.979385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:15.979435 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:15.979644 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:15.980353 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:15.981349 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:15.981665 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:15.981717 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:15.982333 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:15.982334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:15.982335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:15.982339 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:15.982353 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:15.982357 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:15.982405 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:15.982472 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 22:44:15.982484 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3000}
I20260806 22:44:15.994743 1439 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:15.995223 1439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:16.005682 1439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 814d2dc466f74775bcc8bf5f8e835d13: Generated new TSK 0
I20260806 22:44:16.005839 1439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:16.008138 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1308
W20260806 22:44:16.021044 1281 connection.cc:535] client connection to 127.0.0.1:36883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36883): Connection reset by peer (error 104)
2026-08-06T22:44:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:16 dist-test-slave-gjrz krb5kdc[1466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:16 dist-test-slave-gjrz krb5kdc[1466](info): set up 2 sockets
Aug 06 22:44:16 dist-test-slave-gjrz krb5kdc[1466](info): commencing operation
krb5kdc: starting...
W20260806 22:44:17.148165 747 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:17 dist-test-slave-gjrz krb5kdc[1466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:44:17.261610 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:17.342373 1488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:17.342582 1488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:17.342613 1488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:17.343750 1488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:17.343791 1488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:17.343811 1488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:17.343827 1488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:17.343842 1488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:17.345400 1488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:17.345642 1488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:17.345853 1488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:17.350569 1505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:17.350569 1503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:17.350652 1488 server_base.cc:1034] running on GCE node
W20260806 22:44:17.350592 1502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:17.351158 1488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:17.351475 1488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:17.352627 1488 hybrid_clock.cc:648] HybridClock initialized: now 1786056257352612 us; error 29 us; skew 500 ppm
Aug 06 22:44:17 dist-test-slave-gjrz krb5kdc[1466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056257, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:17.355051 1488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:44:17.355480 1488 webserver.cc:460] Webserver started at http://127.0.0.1:43793/ using document root <none> and password file <none>
I20260806 22:44:17.355719 1488 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:17.355764 1488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:17.355854 1488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:17.356822 1488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7"
format_stamp: "Formatted at 2026-08-06 22:44:17 on dist-test-slave-gjrz"
server_key: "17ce33bf362ef16f432f5054830301cb"
server_key_iv: "547a6aabd0fd45f804d4988122f56863"
server_key_version: "encryptionkey@0"
I20260806 22:44:17.357128 1488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7"
format_stamp: "Formatted at 2026-08-06 22:44:17 on dist-test-slave-gjrz"
server_key: "17ce33bf362ef16f432f5054830301cb"
server_key_iv: "547a6aabd0fd45f804d4988122f56863"
server_key_version: "encryptionkey@0"
I20260806 22:44:17.358366 1488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:17.359062 1518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:17.359246 1488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:17.359344 1488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7"
format_stamp: "Formatted at 2026-08-06 22:44:17 on dist-test-slave-gjrz"
server_key: "17ce33bf362ef16f432f5054830301cb"
server_key_iv: "547a6aabd0fd45f804d4988122f56863"
server_key_version: "encryptionkey@0"
I20260806 22:44:17.359411 1488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:17.362565 1488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:17.364593 1488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:17.364717 1488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:17.368432 1488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46701
I20260806 22:44:17.368479 1589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46701 every 8 connection(s)
I20260806 22:44:17.368775 1488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:17.369720 1597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:17.372021 1597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: Bootstrap starting.
I20260806 22:44:17.372558 1597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:17.372822 1597 log.cc:824] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:17.373456 1597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: No bootstrap required, opened a new log
I20260806 22:44:17.375049 1597 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } }
I20260806 22:44:17.375202 1597 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:17.375237 1597 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a8cfc300c34bb1986cbfc9f68efaf7, State: Initialized, Role: FOLLOWER
I20260806 22:44:17.375402 1597 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } }
I20260806 22:44:17.375473 1597 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:17.375505 1597 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:17.375551 1597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:17.375936 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1488
I20260806 22:44:17.376065 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:17.376192 1597 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } }
I20260806 22:44:17.376260 747 external_mini_cluster.cc:1428] Setting key 3de419951c04db4569057a7ea9292be1
I20260806 22:44:17.376299 1597 leader_election.cc:302] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0a8cfc300c34bb1986cbfc9f68efaf7; no voters:
I20260806 22:44:17.376467 1597 leader_election.cc:288] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:17.376490 1603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:17.376616 1603 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [term 1 LEADER]: Becoming Leader. State: Replica: f0a8cfc300c34bb1986cbfc9f68efaf7, State: Running, Role: LEADER
I20260806 22:44:17.376727 1597 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:17.376753 1603 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } }
I20260806 22:44:17.377421 1604 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } } }
I20260806 22:44:17.377661 1604 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:17.377579 1605 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0a8cfc300c34bb1986cbfc9f68efaf7. Latest consensus state: current_term: 1 leader_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a8cfc300c34bb1986cbfc9f68efaf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46701 } } }
I20260806 22:44:17.378217 1605 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7 [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:17.378731 1633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:44:17.378760 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:17.376998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:43865 (local address 127.0.0.1:46701)
0806 22:44:17.377174 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:17.377183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:17.377219 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:44:17.377340 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:17.377347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:17.377380 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:17.377545 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:17.378181 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:17.378185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:17.378185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:17.378190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:17.378271 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:17.378288 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:17.378517 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:17.378552 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:44:17.378594 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":61,"threads_started":1}
W20260806 22:44:17.378798 1633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:17.378916 1631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:17.379117 1631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:17.379673 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:17.380698 1631 catalog_manager.cc:1347] Generated new cluster ID: c9a2139ff86d4070acdccb3781731e14
I20260806 22:44:17.380756 1631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:17.384256 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:17.383345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:55229 (local address 127.0.0.1:46701)
0806 22:44:17.383395 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:17.383396 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:17.383405 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:44:17.383425 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:17.383426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:17.383431 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:17.383489 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:17.384041 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:17.384043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:17.384043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:17.384047 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:17.384062 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:17.384067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:17.384120 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:17.384176 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:44:17.384188 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:44:17.392706 1631 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:17.392855 1631 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:17.399345 1631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0a8cfc300c34bb1986cbfc9f68efaf7: Generated new TSK 0
I20260806 22:44:17.399528 1631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:17.403611 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1488
2026-08-06T22:44:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:17 dist-test-slave-gjrz krb5kdc[1651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:17 dist-test-slave-gjrz krb5kdc[1651](info): set up 2 sockets
Aug 06 22:44:17 dist-test-slave-gjrz krb5kdc[1651](info): commencing operation
krb5kdc: starting...
W20260806 22:44:18.548985 747 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:18.662037 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40141
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:40141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:18.742079 1670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:18.742271 1670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:18.742293 1670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:18.743408 1670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:18.743443 1670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:18.743455 1670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:18.743466 1670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:18.743477 1670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:18.744887 1670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:40141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:18.745074 1670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:18.745240 1670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:18.749778 1678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:18.749804 1679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:18.749780 1681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:18.749794 1670 server_base.cc:1034] running on GCE node
I20260806 22:44:18.750258 1670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:18.750500 1670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:18.751641 1670 hybrid_clock.cc:648] HybridClock initialized: now 1786056258751627 us; error 31 us; skew 500 ppm
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056258, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:18.754060 1670 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:18.754498 1670 webserver.cc:460] Webserver started at http://127.0.186.254:35961/ using document root <none> and password file <none>
I20260806 22:44:18.754690 1670 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:18.754734 1670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:18.754842 1670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:18.755846 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "e8f3307ef0db463bbf17ed018e2cae80"
format_stamp: "Formatted at 2026-08-06 22:44:18 on dist-test-slave-gjrz"
server_key: "7425b22d6602c452bad978d860f53c50"
server_key_iv: "52e3611cd0c66d3fc3f9c193555483c7"
server_key_version: "encryptionkey@0"
I20260806 22:44:18.756186 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "e8f3307ef0db463bbf17ed018e2cae80"
format_stamp: "Formatted at 2026-08-06 22:44:18 on dist-test-slave-gjrz"
server_key: "7425b22d6602c452bad978d860f53c50"
server_key_iv: "52e3611cd0c66d3fc3f9c193555483c7"
server_key_version: "encryptionkey@0"
I20260806 22:44:18.757375 1670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:18.758026 1694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:18.758188 1670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:18.758271 1670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "e8f3307ef0db463bbf17ed018e2cae80"
format_stamp: "Formatted at 2026-08-06 22:44:18 on dist-test-slave-gjrz"
server_key: "7425b22d6602c452bad978d860f53c50"
server_key_iv: "52e3611cd0c66d3fc3f9c193555483c7"
server_key_version: "encryptionkey@0"
I20260806 22:44:18.758332 1670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:18.776823 1670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:18.778838 1670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:18.778968 1670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:18.782809 1670 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:40141
I20260806 22:44:18.782845 1778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:40141 every 8 connection(s)
I20260806 22:44:18.783170 1670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:18.784056 1780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:18.786285 1780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80: Bootstrap starting.
I20260806 22:44:18.786479 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1670
I20260806 22:44:18.786598 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:18.786784 747 external_mini_cluster.cc:1428] Setting key 5e0f98074c28ee7890f352f24adf167a
I20260806 22:44:18.786944 1780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:18.787355 1780 log.cc:824] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:18.788101 1780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80: No bootstrap required, opened a new log
I20260806 22:44:18.790391 1780 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } }
I20260806 22:44:18.790529 1780 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:18.790548 1780 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f3307ef0db463bbf17ed018e2cae80, State: Initialized, Role: FOLLOWER
I20260806 22:44:18.790654 1780 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } }
I20260806 22:44:18.790719 1780 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:18.790742 1780 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:18.790786 1780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:18.791416 1780 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } }
I20260806 22:44:18.791525 1780 leader_election.cc:302] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8f3307ef0db463bbf17ed018e2cae80; no voters:
I20260806 22:44:18.791689 1780 leader_election.cc:288] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:18.791731 1787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:18.791908 1787 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [term 1 LEADER]: Becoming Leader. State: Replica: e8f3307ef0db463bbf17ed018e2cae80, State: Running, Role: LEADER
I20260806 22:44:18.791945 1780 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:18.792073 1787 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } }
I20260806 22:44:18.792376 1787 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8f3307ef0db463bbf17ed018e2cae80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } } }
I20260806 22:44:18.792440 1787 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:18.792584 1787 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8f3307ef0db463bbf17ed018e2cae80. Latest consensus state: current_term: 1 leader_uuid: "e8f3307ef0db463bbf17ed018e2cae80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f3307ef0db463bbf17ed018e2cae80" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40141 } } }
I20260806 22:44:18.792630 1787 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:18.793030 1794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:18.793534 1794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:18.795106 1794 catalog_manager.cc:1347] Generated new cluster ID: dd4e05749ae44d90b7df33f10fb67bdf
I20260806 22:44:18.795159 1794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:18.798827 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:18.787256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58113 (local address 127.0.186.254:40141)
0806 22:44:18.787450 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:18.787459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:18.787493 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:44:18.788017 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:18.788027 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:18.788087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:18.788231 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:18.788826 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:18.790236 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:18.791102 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:18.791738 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:18.791850 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:18.794503 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:18.794507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:18.794508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:18.794519 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:18.796616 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:18.797102 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:18.797106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:18.797106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:18.797216 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:18.797488 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:18.797489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:18.797489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:18.797791 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:18.797958 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:18.798494 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 22:44:18.798604 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260806 22:44:18.799826 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:18.808327 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:18.801250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49773 (local address 127.0.186.254:40141)
0806 22:44:18.801291 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:18.801293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:18.801395 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:44:18.801501 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:18.801502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:18.801512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:18.801542 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:18.802223 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:18.802960 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:18.803994 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:18.804311 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:18.804352 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:18.805717 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:18.805719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:18.805719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:18.805726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:18.807045 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:18.807494 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:18.807495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:18.807495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:18.807550 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:18.807773 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:18.807773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:18.807773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:18.807890 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:18.807959 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:18.808178 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:44:18.808238 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260806 22:44:18.813823 1794 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:18.814275 1794 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:18.819918 1794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8f3307ef0db463bbf17ed018e2cae80: Generated new TSK 0
I20260806 22:44:18.820070 1794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:18.822625 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1670
W20260806 22:44:18.841545 1648 connection.cc:535] client connection to 127.0.186.254:40141 recv error: Network error: failed to read from TLS socket (remote: 127.0.186.254:40141): Connection reset by peer (error 104)
2026-08-06T22:44:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1822](info): set up 2 sockets
Aug 06 22:44:18 dist-test-slave-gjrz krb5kdc[1822](info): commencing operation
krb5kdc: starting...
W20260806 22:44:19.956753 747 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:20 dist-test-slave-gjrz krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:20.071146 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45867
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:45867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:20.151248 1845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:20.151479 1845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:20.151510 1845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:20.152633 1845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:20.152673 1845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:20.152691 1845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:20.152707 1845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:20.152722 1845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:20.154186 1845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:45867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:20.154430 1845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:20.154635 1845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:20.159121 1856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:20.159216 1851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:20.159130 1852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:20.159153 1845 server_base.cc:1034] running on GCE node
I20260806 22:44:20.159584 1845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:20.159849 1845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:20.160987 1845 hybrid_clock.cc:648] HybridClock initialized: now 1786056260160972 us; error 28 us; skew 500 ppm
Aug 06 22:44:20 dist-test-slave-gjrz krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:20.163354 1845 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:20.163749 1845 webserver.cc:460] Webserver started at http://127.0.186.254:33571/ using document root <none> and password file <none>
I20260806 22:44:20.163954 1845 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:20.164002 1845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:20.164132 1845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:20.165133 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "e936fb9ca6de484696313d42b0eb3034"
format_stamp: "Formatted at 2026-08-06 22:44:20 on dist-test-slave-gjrz"
server_key: "a58cabd98f91a408aa80d168b543d8c7"
server_key_iv: "e69adeec96d50242279aaff9ed18eb12"
server_key_version: "encryptionkey@0"
I20260806 22:44:20.165496 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "e936fb9ca6de484696313d42b0eb3034"
format_stamp: "Formatted at 2026-08-06 22:44:20 on dist-test-slave-gjrz"
server_key: "a58cabd98f91a408aa80d168b543d8c7"
server_key_iv: "e69adeec96d50242279aaff9ed18eb12"
server_key_version: "encryptionkey@0"
I20260806 22:44:20.166697 1845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:20.167377 1865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:20.167517 1845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:20.167600 1845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "e936fb9ca6de484696313d42b0eb3034"
format_stamp: "Formatted at 2026-08-06 22:44:20 on dist-test-slave-gjrz"
server_key: "a58cabd98f91a408aa80d168b543d8c7"
server_key_iv: "e69adeec96d50242279aaff9ed18eb12"
server_key_version: "encryptionkey@0"
I20260806 22:44:20.167670 1845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:20.180219 1845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:20.182180 1845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:20.182300 1845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:20.186082 1845 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:45867
I20260806 22:44:20.186125 1954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:45867 every 8 connection(s)
I20260806 22:44:20.186416 1845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:20.187244 1955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:20.189411 1955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034: Bootstrap starting.
I20260806 22:44:20.189936 1955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:20.190172 1955 log.cc:824] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:20.190776 1955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034: No bootstrap required, opened a new log
I20260806 22:44:20.192401 1955 raft_consensus.cc:348] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } }
I20260806 22:44:20.192531 1955 raft_consensus.cc:374] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:20.192553 1955 raft_consensus.cc:729] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e936fb9ca6de484696313d42b0eb3034, State: Initialized, Role: FOLLOWER
I20260806 22:44:20.192662 1955 consensus_queue.cc:260] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } }
I20260806 22:44:20.192734 1955 raft_consensus.cc:388] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:20.192761 1955 raft_consensus.cc:482] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:20.192792 1955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:20.193266 1955 raft_consensus.cc:504] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } }
I20260806 22:44:20.193348 1955 leader_election.cc:302] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e936fb9ca6de484696313d42b0eb3034; no voters:
I20260806 22:44:20.193466 1955 leader_election.cc:288] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:20.193507 1960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:20.193678 1955 sys_catalog.cc:564] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:20.193699 1960 raft_consensus.cc:686] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [term 1 LEADER]: Becoming Leader. State: Replica: e936fb9ca6de484696313d42b0eb3034, State: Running, Role: LEADER
I20260806 22:44:20.193781 1960 consensus_queue.cc:237] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } }
I20260806 22:44:20.194160 1961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e936fb9ca6de484696313d42b0eb3034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } } }
I20260806 22:44:20.194243 1961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:20.194373 1963 sys_catalog.cc:455] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e936fb9ca6de484696313d42b0eb3034. Latest consensus state: current_term: 1 leader_uuid: "e936fb9ca6de484696313d42b0eb3034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e936fb9ca6de484696313d42b0eb3034" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45867 } } }
I20260806 22:44:20.194485 1963 sys_catalog.cc:458] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:20.194463 1972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:20.195134 1972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:20.195742 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 1845
I20260806 22:44:20.195858 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:20.196065 747 external_mini_cluster.cc:1428] Setting key 8fa681f3a5bb8e2280aafb429f69f2ed
I20260806 22:44:20.196825 1972 catalog_manager.cc:1347] Generated new cluster ID: bffbc8d0021c430c8dbfc4656beade93
I20260806 22:44:20.196879 1972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:20.201932 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:20.196472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58627 (local address 127.0.186.254:45867)
0806 22:44:20.196903 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:20.196911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:20.196942 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:44:20.197063 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:20.197069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:20.197121 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:20.197242 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:20.197479 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:20.198834 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:20.199928 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:20.200497 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:20.200594 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:20.201265 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:20.201269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:20.201269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:20.201455 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:20.201495 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:20.201509 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:20.201593 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:20.201691 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:44:20.201720 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":65,"threads_started":1}
I20260806 22:44:20.202754 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:20.207481 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:20.204134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33529 (local address 127.0.186.254:45867)
0806 22:44:20.204178 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:20.204179 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:20.204259 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:44:20.204331 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:20.204332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:20.204352 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:20.204378 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:20.204562 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:20.205251 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:20.206211 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:20.206537 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:20.206577 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:20.207221 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:20.207223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:20.207223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:20.207227 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:20.207240 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:20.207243 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:20.207292 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:20.207403 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:44:20.207415 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:20.224852 1972 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:20.225339 1972 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:20.229249 1972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e936fb9ca6de484696313d42b0eb3034: Generated new TSK 0
I20260806 22:44:20.229398 1972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:20.231235 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 1845
W20260806 22:44:20.244460 1819 connection.cc:535] client connection to 127.0.186.254:45867 recv error: Network error: failed to read from TLS socket (remote: 127.0.186.254:45867): Connection reset by peer (error 104)
2026-08-06T22:44:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:20 dist-test-slave-gjrz krb5kdc[1999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:20 dist-test-slave-gjrz krb5kdc[1999](info): set up 2 sockets
Aug 06 22:44:20 dist-test-slave-gjrz krb5kdc[1999](info): commencing operation
krb5kdc: starting...
W20260806 22:44:21.361359 747 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:21 dist-test-slave-gjrz krb5kdc[1999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:21.475184 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38585
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:38585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:21.565056 2020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:21.565258 2020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:21.565280 2020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:21.566435 2020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:21.566480 2020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:21.566493 2020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:21.566504 2020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:21.566515 2020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:21.568070 2020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:38585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:38585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:21.568310 2020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:21.568528 2020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:21.573573 2020 server_base.cc:1034] running on GCE node
W20260806 22:44:21.573503 2027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:21.573489 2030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:21.573494 2028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:21.573899 2020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:21.574239 2020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:21.575387 2020 hybrid_clock.cc:648] HybridClock initialized: now 1786056261575366 us; error 34 us; skew 500 ppm
Aug 06 22:44:21 dist-test-slave-gjrz krb5kdc[1999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:21.577782 2020 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:21.578240 2020 webserver.cc:460] Webserver started at http://127.0.186.254:41727/ using document root <none> and password file <none>
I20260806 22:44:21.578466 2020 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:21.578516 2020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:21.578644 2020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:21.579747 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "4d0b5d68237244029bf0796a36caf3f8"
format_stamp: "Formatted at 2026-08-06 22:44:21 on dist-test-slave-gjrz"
server_key: "38887911ee5a32829b01ae11892ddba2"
server_key_iv: "f9dc1d4bde943d2ddb81e1b53ced334f"
server_key_version: "encryptionkey@0"
I20260806 22:44:21.580096 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "4d0b5d68237244029bf0796a36caf3f8"
format_stamp: "Formatted at 2026-08-06 22:44:21 on dist-test-slave-gjrz"
server_key: "38887911ee5a32829b01ae11892ddba2"
server_key_iv: "f9dc1d4bde943d2ddb81e1b53ced334f"
server_key_version: "encryptionkey@0"
I20260806 22:44:21.581322 2020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:21.581995 2041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:21.582178 2020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:21.582265 2020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "4d0b5d68237244029bf0796a36caf3f8"
format_stamp: "Formatted at 2026-08-06 22:44:21 on dist-test-slave-gjrz"
server_key: "38887911ee5a32829b01ae11892ddba2"
server_key_iv: "f9dc1d4bde943d2ddb81e1b53ced334f"
server_key_version: "encryptionkey@0"
I20260806 22:44:21.582327 2020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:21.585634 2020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:21.587473 2020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:21.587601 2020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:21.591480 2020 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:38585
I20260806 22:44:21.591528 2132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:38585 every 8 connection(s)
I20260806 22:44:21.591813 2020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:21.592773 2133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:21.595007 2133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8: Bootstrap starting.
I20260806 22:44:21.595573 2133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:21.595817 2133 log.cc:824] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:21.596431 2133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8: No bootstrap required, opened a new log
I20260806 22:44:21.598052 2133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } }
I20260806 22:44:21.598186 2133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:21.598206 2133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0b5d68237244029bf0796a36caf3f8, State: Initialized, Role: FOLLOWER
I20260806 22:44:21.598307 2133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } }
I20260806 22:44:21.598357 2133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:21.598376 2133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:21.598404 2133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:21.598884 2133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } }
I20260806 22:44:21.598975 2133 leader_election.cc:302] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d0b5d68237244029bf0796a36caf3f8; no voters:
I20260806 22:44:21.599094 2133 leader_election.cc:288] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:21.599144 2139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:21.599323 2139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 4d0b5d68237244029bf0796a36caf3f8, State: Running, Role: LEADER
I20260806 22:44:21.599375 2133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:21.599414 2139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } }
I20260806 22:44:21.599620 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2020
I20260806 22:44:21.599737 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:21.599932 747 external_mini_cluster.cc:1428] Setting key 12a2533bc47018a8b12b843ba307f188
I20260806 22:44:21.600132 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d0b5d68237244029bf0796a36caf3f8. Latest consensus state: current_term: 1 leader_uuid: "4d0b5d68237244029bf0796a36caf3f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } } }
I20260806 22:44:21.600229 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:21.600723 2141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d0b5d68237244029bf0796a36caf3f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d0b5d68237244029bf0796a36caf3f8" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 38585 } } }
I20260806 22:44:21.600811 2141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:21.600894 2155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:21.601420 2155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:21.602533 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:21.600551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46743 (local address 127.0.186.254:38585)
0806 22:44:21.600995 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:21.601004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:21.601032 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:44:21.601146 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:21.601153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:21.601185 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:21.601322 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:21.602002 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:21.602006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:21.602006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:21.602010 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:21.602067 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:21.602081 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:21.602134 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:21.602257 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:44:21.602334 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":89,"threads_started":1}
I20260806 22:44:21.603122 2155 catalog_manager.cc:1347] Generated new cluster ID: 712c4b001f74406092babec4768334ab
I20260806 22:44:21.603171 2155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:21.607772 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:21.610693 2155 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:21.610839 2155 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:21.611627 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:21.610453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60427 (local address 127.0.186.254:38585)
0806 22:44:21.610493 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:21.610495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:21.610651 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:44:21.610784 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:21.610785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:21.610791 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:21.610848 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:21.611411 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:21.611412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:21.611413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:21.611417 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:21.611431 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:21.611435 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:21.611487 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:21.611538 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:44:21.611549 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:44:21.616279 2155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d0b5d68237244029bf0796a36caf3f8: Generated new TSK 0
I20260806 22:44:21.616444 2155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:21.618014 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2020
2026-08-06T22:44:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:21 dist-test-slave-gjrz krb5kdc[2185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:21 dist-test-slave-gjrz krb5kdc[2185](info): set up 2 sockets
Aug 06 22:44:21 dist-test-slave-gjrz krb5kdc[2185](info): commencing operation
krb5kdc: starting...
W20260806 22:44:22.752555 747 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:22 dist-test-slave-gjrz krb5kdc[2185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:22.867738 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37577
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:37577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:22.948853 2201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:22.949057 2201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:22.949086 2201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:22.950242 2201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:22.950285 2201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:22.950304 2201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:22.950318 2201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:22.950335 2201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:22.951871 2201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:37577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:37577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:22.952121 2201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:22.952317 2201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:22.956794 2209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:22.956828 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:22.956835 2211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:22.956897 2201 server_base.cc:1034] running on GCE node
I20260806 22:44:22.957247 2201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:22.957525 2201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:22.958673 2201 hybrid_clock.cc:648] HybridClock initialized: now 1786056262958651 us; error 33 us; skew 500 ppm
Aug 06 22:44:22 dist-test-slave-gjrz krb5kdc[2185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:22.961097 2201 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:22.961537 2201 webserver.cc:460] Webserver started at http://127.0.186.254:42903/ using document root <none> and password file <none>
I20260806 22:44:22.961750 2201 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:22.961799 2201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:22.961936 2201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:22.962940 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "98d9474bfc9e4d959777f43bb851996c"
format_stamp: "Formatted at 2026-08-06 22:44:22 on dist-test-slave-gjrz"
server_key: "0166ada10847bfb9ca094795be5b12d9"
server_key_iv: "4499563186dbac430faf269bd8ba43fe"
server_key_version: "encryptionkey@0"
I20260806 22:44:22.963307 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "98d9474bfc9e4d959777f43bb851996c"
format_stamp: "Formatted at 2026-08-06 22:44:22 on dist-test-slave-gjrz"
server_key: "0166ada10847bfb9ca094795be5b12d9"
server_key_iv: "4499563186dbac430faf269bd8ba43fe"
server_key_version: "encryptionkey@0"
I20260806 22:44:22.964536 2201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:22.965222 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:22.965376 2201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:22.965464 2201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "98d9474bfc9e4d959777f43bb851996c"
format_stamp: "Formatted at 2026-08-06 22:44:22 on dist-test-slave-gjrz"
server_key: "0166ada10847bfb9ca094795be5b12d9"
server_key_iv: "4499563186dbac430faf269bd8ba43fe"
server_key_version: "encryptionkey@0"
I20260806 22:44:22.965530 2201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:22.980711 2201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:22.982563 2201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:22.982690 2201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:22.986512 2201 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:37577
I20260806 22:44:22.986557 2299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:37577 every 8 connection(s)
I20260806 22:44:22.986867 2201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:22.987708 2304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:22.989904 2304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: Bootstrap starting.
I20260806 22:44:22.990455 2304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:22.990717 2304 log.cc:824] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:22.991386 2304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: No bootstrap required, opened a new log
I20260806 22:44:22.992422 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2201
I20260806 22:44:22.992511 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:22.992686 747 external_mini_cluster.cc:1428] Setting key 2b4c878b226d9593e0236dbf947138f3
I20260806 22:44:22.993166 2304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } }
I20260806 22:44:22.993342 2304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:22.993391 2304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d9474bfc9e4d959777f43bb851996c, State: Initialized, Role: FOLLOWER
I20260806 22:44:22.993510 2304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } }
I20260806 22:44:22.993587 2304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:22.993623 2304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:22.993669 2304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:22.994351 2304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } }
I20260806 22:44:22.994462 2304 leader_election.cc:302] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d9474bfc9e4d959777f43bb851996c; no voters:
I20260806 22:44:22.994611 2304 leader_election.cc:288] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:22.994645 2315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:22.994796 2315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [term 1 LEADER]: Becoming Leader. State: Replica: 98d9474bfc9e4d959777f43bb851996c, State: Running, Role: LEADER
I20260806 22:44:22.994855 2304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:22.994951 2315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } }
I20260806 22:44:22.995316 2315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d9474bfc9e4d959777f43bb851996c. Latest consensus state: current_term: 1 leader_uuid: "98d9474bfc9e4d959777f43bb851996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } } }
I20260806 22:44:22.995380 2317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d9474bfc9e4d959777f43bb851996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9474bfc9e4d959777f43bb851996c" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 37577 } } }
I20260806 22:44:22.995472 2315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:22.995479 2317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:22.996803 2345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:22.996858 2345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:22.996909 2331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:22.997095 2331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:22.998678 2331 catalog_manager.cc:1347] Generated new cluster ID: edd311046aa448afb0c54d58894d615c
I20260806 22:44:22.998762 2331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:22 dist-test-slave-gjrz krb5kdc[2185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:23.003252 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:22.993156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45383 (local address 127.0.186.254:37577)
0806 22:44:22.993360 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:22.993369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:22.993403 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:44:22.993524 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:22.993531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:22.993595 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:22.993735 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:22.994303 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:22.995621 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:22.996617 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:22.997201 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:22.997316 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:22.999901 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:22.999905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:22.999905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:22.999916 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:23.001725 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:23.002132 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:23.002134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:23.002134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:23.002243 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:23.002438 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:23.002438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:23.002438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:23.002713 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:23.002857 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:23.002950 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:44:23.003076 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260806 22:44:23.004180 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:23.009915 2331 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:23.010568 2331 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:23.012382 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:23.005751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42189 (local address 127.0.186.254:37577)
0806 22:44:23.005780 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:23.005781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:23.005789 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:23.005845 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:23.005846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:23.005853 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:23.005899 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:23.006497 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:23.007196 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:23.008185 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:23.008477 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:23.008528 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:23.009774 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:23.009775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:23.009775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:23.009782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:23.011354 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:23.011713 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:23.011713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:23.011713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:23.011765 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:23.011933 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:23.011934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:23.011934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:23.012034 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:23.012122 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:23.012213 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:44:23.012298 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:44:23.022032 2331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98d9474bfc9e4d959777f43bb851996c: Generated new TSK 0
I20260806 22:44:23.022202 2331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:23.026623 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2201
2026-08-06T22:44:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:23 dist-test-slave-gjrz krb5kdc[2358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:23 dist-test-slave-gjrz krb5kdc[2358](info): set up 2 sockets
Aug 06 22:44:23 dist-test-slave-gjrz krb5kdc[2358](info): commencing operation
krb5kdc: starting...
W20260806 22:44:24.160212 747 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:24 dist-test-slave-gjrz krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:24.272360 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45479
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:24.353336 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:24.353536 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:24.353559 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:24.354678 2383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:24.354719 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:24.354733 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:24.354746 2383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:24.354770 2383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:24.356277 2383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:24.356508 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:24.356714 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:24.361343 2383 server_base.cc:1034] running on GCE node
W20260806 22:44:24.361303 2390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:24.361296 2391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:24.361303 2393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:24.361766 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:24.362082 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:24.363246 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786056264363231 us; error 29 us; skew 500 ppm
Aug 06 22:44:24 dist-test-slave-gjrz krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:24.365597 2383 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:24.366006 2383 webserver.cc:460] Webserver started at http://127.0.186.254:41535/ using document root <none> and password file <none>
I20260806 22:44:24.366235 2383 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:24.366279 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:24.366402 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:24.367444 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "a7de3894ad3346d7b053124ec03be1a4"
format_stamp: "Formatted at 2026-08-06 22:44:24 on dist-test-slave-gjrz"
server_key: "d0814ae6904db5eba0afa147a80b316f"
server_key_iv: "89cad2a90d6e14bcf9343f23f359734a"
server_key_version: "encryptionkey@0"
I20260806 22:44:24.367767 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "a7de3894ad3346d7b053124ec03be1a4"
format_stamp: "Formatted at 2026-08-06 22:44:24 on dist-test-slave-gjrz"
server_key: "d0814ae6904db5eba0afa147a80b316f"
server_key_iv: "89cad2a90d6e14bcf9343f23f359734a"
server_key_version: "encryptionkey@0"
I20260806 22:44:24.368942 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:24.369663 2404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:24.369799 2383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:24.369884 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "a7de3894ad3346d7b053124ec03be1a4"
format_stamp: "Formatted at 2026-08-06 22:44:24 on dist-test-slave-gjrz"
server_key: "d0814ae6904db5eba0afa147a80b316f"
server_key_iv: "89cad2a90d6e14bcf9343f23f359734a"
server_key_version: "encryptionkey@0"
I20260806 22:44:24.369943 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:24.388656 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:24.390489 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:24.390616 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:24.394523 2383 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:45479
I20260806 22:44:24.394569 2490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:45479 every 8 connection(s)
I20260806 22:44:24.394878 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:24.395725 2491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:24.397161 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2383
I20260806 22:44:24.397269 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:24.397466 747 external_mini_cluster.cc:1428] Setting key faab60ccba679fc18a858b6d82211b45
I20260806 22:44:24.398224 2491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4: Bootstrap starting.
I20260806 22:44:24.398826 2491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:24.399138 2491 log.cc:824] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:24.399787 2491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4: No bootstrap required, opened a new log
I20260806 22:44:24.402047 2491 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } }
I20260806 22:44:24.402204 2491 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:24.402228 2491 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7de3894ad3346d7b053124ec03be1a4, State: Initialized, Role: FOLLOWER
I20260806 22:44:24.402338 2491 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } }
I20260806 22:44:24.402411 2491 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:24.402452 2491 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:24.402504 2491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:24.402756 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:24.397964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59789 (local address 127.0.186.254:45479)
0806 22:44:24.398138 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:24.398146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:24.398171 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:44:24.398611 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:24.398618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:24.398665 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:24.398776 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:24.399027 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:24.399993 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:24.401016 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:24.401409 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:24.401487 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:24.402114 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:24.402118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:24.402118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:24.402287 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:24.402323 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:24.402338 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:24.402392 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:24.402494 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:44:24.402572 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260806 22:44:24.403137 2491 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } }
I20260806 22:44:24.403239 2491 leader_election.cc:302] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7de3894ad3346d7b053124ec03be1a4; no voters:
I20260806 22:44:24.403417 2491 leader_election.cc:288] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:24.403455 2496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:24.403551 2496 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [term 1 LEADER]: Becoming Leader. State: Replica: a7de3894ad3346d7b053124ec03be1a4, State: Running, Role: LEADER
I20260806 22:44:24.403672 2491 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:24.403632 2496 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } }
I20260806 22:44:24.403776 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:24.404026 2497 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7de3894ad3346d7b053124ec03be1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } } }
I20260806 22:44:24.404201 2497 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:24.404145 2498 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7de3894ad3346d7b053124ec03be1a4. Latest consensus state: current_term: 1 leader_uuid: "a7de3894ad3346d7b053124ec03be1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7de3894ad3346d7b053124ec03be1a4" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 45479 } } }
I20260806 22:44:24.404373 2498 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:24.404707 2506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:24.405222 2506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:24.407851 2506 catalog_manager.cc:1347] Generated new cluster ID: 6012e640cd8e40e7a23d7bbe4298e855
I20260806 22:44:24.407895 2506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:24.408654 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:24.405342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48013 (local address 127.0.186.254:45479)
0806 22:44:24.405371 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:24.405373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:24.405477 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:44:24.405563 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:24.405564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:24.405572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:24.405600 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:24.405787 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:24.406483 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:24.407467 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:24.407728 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:24.407784 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:24.408413 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:24.408414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:24.408415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:24.408419 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:24.408433 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:24.408437 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:24.408464 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:24.408562 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:44:24.408585 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:44:24.429986 2506 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:24.430784 2506 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:24.436543 2506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7de3894ad3346d7b053124ec03be1a4: Generated new TSK 0
I20260806 22:44:24.436720 2506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:24.447682 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2383
2026-08-06T22:44:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:24 dist-test-slave-gjrz krb5kdc[2548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:24 dist-test-slave-gjrz krb5kdc[2548](info): set up 2 sockets
Aug 06 22:44:24 dist-test-slave-gjrz krb5kdc[2548](info): commencing operation
krb5kdc: starting...
W20260806 22:44:25.576965 747 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:25 dist-test-slave-gjrz krb5kdc[2548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:25.691601 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40181
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:39049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:25.772356 2565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:25.772573 2565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:25.772605 2565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:25.773736 2565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:25.773775 2565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:25.773794 2565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:25.773810 2565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:25.773825 2565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:25.775316 2565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:39049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:25.775564 2565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:25.775764 2565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:25.780321 2574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:25.780329 2573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:25.780360 2565 server_base.cc:1034] running on GCE node
W20260806 22:44:25.780468 2576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:25.780690 2565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:25.780957 2565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:25.782094 2565 hybrid_clock.cc:648] HybridClock initialized: now 1786056265782081 us; error 29 us; skew 500 ppm
Aug 06 22:44:25 dist-test-slave-gjrz krb5kdc[2548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:25.784511 2565 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:25.784891 2565 webserver.cc:460] Webserver started at http://127.0.186.254:41289/ using document root <none> and password file <none>
I20260806 22:44:25.785084 2565 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:25.785125 2565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:25.785214 2565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:25.786196 2565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "05762dcb1a4e438faf02b9c07c87f52d"
format_stamp: "Formatted at 2026-08-06 22:44:25 on dist-test-slave-gjrz"
server_key: "39b3e4e97c3db186760dfa6495d20e55"
server_key_iv: "868574c817471d96d7fcd146fd1f5772"
server_key_version: "encryptionkey@0"
I20260806 22:44:25.786492 2565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "05762dcb1a4e438faf02b9c07c87f52d"
format_stamp: "Formatted at 2026-08-06 22:44:25 on dist-test-slave-gjrz"
server_key: "39b3e4e97c3db186760dfa6495d20e55"
server_key_iv: "868574c817471d96d7fcd146fd1f5772"
server_key_version: "encryptionkey@0"
I20260806 22:44:25.787693 2565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:25.788349 2586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:25.788506 2565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:25.788592 2565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "05762dcb1a4e438faf02b9c07c87f52d"
format_stamp: "Formatted at 2026-08-06 22:44:25 on dist-test-slave-gjrz"
server_key: "39b3e4e97c3db186760dfa6495d20e55"
server_key_iv: "868574c817471d96d7fcd146fd1f5772"
server_key_version: "encryptionkey@0"
I20260806 22:44:25.788658 2565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:25.791774 2565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:25.793819 2565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:25.793972 2565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:25.797734 2565 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:40181
I20260806 22:44:25.797777 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:40181 every 8 connection(s)
I20260806 22:44:25.798062 2565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:25.798995 2672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:25.801148 2672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d: Bootstrap starting.
I20260806 22:44:25.801657 2672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:25.801918 2672 log.cc:824] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:25.802510 2672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d: No bootstrap required, opened a new log
I20260806 22:44:25.804149 2672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } }
I20260806 22:44:25.804278 2672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:25.804301 2672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05762dcb1a4e438faf02b9c07c87f52d, State: Initialized, Role: FOLLOWER
I20260806 22:44:25.804404 2672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } }
I20260806 22:44:25.804455 2672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:25.804479 2672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:25.804508 2672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:25.804956 2672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } }
I20260806 22:44:25.805049 2672 leader_election.cc:302] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05762dcb1a4e438faf02b9c07c87f52d; no voters:
I20260806 22:44:25.805176 2672 leader_election.cc:288] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:25.805235 2681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:25.805399 2672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:25.805560 2681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [term 1 LEADER]: Becoming Leader. State: Replica: 05762dcb1a4e438faf02b9c07c87f52d, State: Running, Role: LEADER
I20260806 22:44:25.805657 2681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } }
I20260806 22:44:25.806026 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05762dcb1a4e438faf02b9c07c87f52d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } } }
I20260806 22:44:25.806036 2683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05762dcb1a4e438faf02b9c07c87f52d. Latest consensus state: current_term: 1 leader_uuid: "05762dcb1a4e438faf02b9c07c87f52d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05762dcb1a4e438faf02b9c07c87f52d" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 40181 } } }
I20260806 22:44:25.806119 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:25.806141 2683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:25.806277 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2565
I20260806 22:44:25.806393 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:25.806589 747 external_mini_cluster.cc:1428] Setting key 1399cec356179bac5c27d04ebff8247f
I20260806 22:44:25.806830 2692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:25.807545 2692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:25.808650 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:25.806995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59487 (local address 127.0.186.254:40181)
0806 22:44:25.807153 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:25.807162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:25.807198 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:44:25.807402 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:25.807410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:25.807440 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:25.807847 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:25.808256 (+ 409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:25.808260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:25.808260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:25.808264 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:25.808303 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:25.808316 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:25.808384 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:25.808446 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:44:25.808484 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260806 22:44:25.809476 2692 catalog_manager.cc:1347] Generated new cluster ID: 25f2771f447d4f528f3d69404d8a0130
I20260806 22:44:25.809528 2692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:25.809659 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:25.812486 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:25.810943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43899 (local address 127.0.186.254:40181)
0806 22:44:25.810980 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:25.810982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:25.811437 (+ 455us) server_negotiation.cc:407] Connection header received
0806 22:44:25.811533 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:25.811534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:25.811538 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:25.811566 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:25.812246 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:25.812247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:25.812248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:25.812252 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:25.812266 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:25.812270 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:25.812297 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:25.812413 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:44:25.812426 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:44:25.819104 2692 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:25.819257 2692 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:25.822126 2692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05762dcb1a4e438faf02b9c07c87f52d: Generated new TSK 0
I20260806 22:44:25.822283 2692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:25.824365 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2565
2026-08-06T22:44:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1378 ms)
[----------] 18 tests from AuthTokenIssuingTest (25272 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:25 dist-test-slave-gjrz krb5kdc[2719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:25 dist-test-slave-gjrz krb5kdc[2719](info): set up 2 sockets
Aug 06 22:44:25 dist-test-slave-gjrz krb5kdc[2719](info): commencing operation
krb5kdc: starting...
W20260806 22:44:26.951992 747 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:27.067039 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:44613
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:27.149192 2742 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:27.149406 2742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:27.149422 2742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:27.149441 2742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:27.150513 2742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:27.150548 2742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:27.150560 2742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:27.150571 2742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:27.150581 2742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:27.152118 2742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:27.152328 2742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:27.152498 2742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:27.157155 2751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.157123 2748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.157135 2753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:27.157498 2742 server_base.cc:1034] running on GCE node
I20260806 22:44:27.157716 2742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:27.158003 2742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:27.159157 2742 hybrid_clock.cc:648] HybridClock initialized: now 1786056267159140 us; error 31 us; skew 500 ppm
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:27.165083 2742 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:27.165534 2742 webserver.cc:460] Webserver started at http://127.0.186.254:44045/ using document root <none> and password file <none>
I20260806 22:44:27.165743 2742 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:27.165792 2742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:27.165916 2742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:27.166949 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "697f2ca682bd401b8dce643aec1c39fd"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "9feaa2510b3d5d6a639bb496481f1a45"
server_key_iv: "a0c008ce18c13a525da9d46158bf4fcc"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.167363 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "697f2ca682bd401b8dce643aec1c39fd"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "9feaa2510b3d5d6a639bb496481f1a45"
server_key_iv: "a0c008ce18c13a525da9d46158bf4fcc"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.168629 2742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:27.169385 2763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:27.169546 2742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:27.169632 2742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "697f2ca682bd401b8dce643aec1c39fd"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "9feaa2510b3d5d6a639bb496481f1a45"
server_key_iv: "a0c008ce18c13a525da9d46158bf4fcc"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.169700 2742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:27.197530 2742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:27.199548 2742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:27.199682 2742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:27.203742 2742 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:44613
I20260806 22:44:27.203787 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:44613 every 8 connection(s)
I20260806 22:44:27.204144 2742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:27.205098 2844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:27.206661 2844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:27.211879 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2742
I20260806 22:44:27.211985 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:27.212138 747 external_mini_cluster.cc:1428] Setting key b5c0887b2117774049b19ebc6235306f
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:27.216041 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.207334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.0.186.254:44613)
0806 22:44:27.207567 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.207570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.207611 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:44:27.208143 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.208144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.208160 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.208230 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.208708 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.209498 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.210308 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.210607 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.210677 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.212792 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.212800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.212800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.212807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.214442 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.214923 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.214927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.214927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.215043 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.215247 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.215248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.215248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.215553 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.215637 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.215800 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.215949 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260806 22:44:27.216048 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.207321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44613 (local address 127.0.0.1:35784)
0806 22:44:27.207507 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.207543 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.208024 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.208248 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.208250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.208486 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.208642 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.208646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.209573 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.209574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.210249 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.210251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.210683 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.210684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.210742 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.211199 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.211207 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.212689 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.214538 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.214545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.214546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.214847 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.215110 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.215111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.215111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.215191 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.215617 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.215622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.215677 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.215748 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.215869 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260806 22:44:27.216966 2844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:27.217303 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.217179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34182)
0806 22:44:27.217224 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.217279 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:27.217447 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
I20260806 22:44:27.217854 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.217774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34192)
0806 22:44:27.217813 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.217838 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:27.217952 2844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:41945: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
I20260806 22:44:27.236235 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:41945
--webserver_interface=127.0.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:27.267179 2844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } attempt: 1
I20260806 22:44:27.267726 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.267586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34208)
0806 22:44:27.267643 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.267688 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:44:27.268117 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.268020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34224)
0806 22:44:27.268064 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.268094 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:44:27.268249 2844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:41945: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
W20260806 22:44:27.317502 2853 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:27.317703 2853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:27.317719 2853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:27.317737 2853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:27.318858 2853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:27.318902 2853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:27.318922 2853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:27.318933 2853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:27.318944 2853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:27.320899 2853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:27.321152 2853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:27.321359 2853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:27.326030 2864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.326076 2860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.326040 2859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:27.326236 2853 server_base.cc:1034] running on GCE node
I20260806 22:44:27.326540 2853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:27.326812 2853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:27.327957 2853 hybrid_clock.cc:648] HybridClock initialized: now 1786056267327932 us; error 37 us; skew 500 ppm
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:27.330330 2853 init.cc:377] Logged in from keytab as kudu/127.0.186.253@KRBTEST.COM (short username kudu)
I20260806 22:44:27.330750 2853 webserver.cc:460] Webserver started at http://127.0.186.253:46707/ using document root <none> and password file <none>
I20260806 22:44:27.330947 2853 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:27.330994 2853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:27.331118 2853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:27.332175 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data/instance:
uuid: "6c541324d60447cb96d30e25c5c8bc8d"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "ac675f692e961ef8bafe51471b28db78"
server_key_iv: "22a2ecfa91e10862b34772826aa4e4c3"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.332518 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal/instance:
uuid: "6c541324d60447cb96d30e25c5c8bc8d"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "ac675f692e961ef8bafe51471b28db78"
server_key_iv: "22a2ecfa91e10862b34772826aa4e4c3"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.333761 2853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:27.334414 2878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:27.334568 2853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:27.334650 2853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "6c541324d60447cb96d30e25c5c8bc8d"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "ac675f692e961ef8bafe51471b28db78"
server_key_iv: "22a2ecfa91e10862b34772826aa4e4c3"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.334717 2853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:27.336410 2844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } attempt: 2
I20260806 22:44:27.336795 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.336678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34236)
0806 22:44:27.336733 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.336765 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:44:27.337116 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.337026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34252)
0806 22:44:27.337066 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.337091 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:44:27.337229 2844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:41945: Network error: Client connection negotiation failed: client connection to 127.0.186.253:41945: connect: Connection refused (error 111)
I20260806 22:44:27.355628 2853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:27.357407 2853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:27.357559 2853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:27.361275 2853 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.253:41945
I20260806 22:44:27.361311 2958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.253:41945 every 8 connection(s)
I20260806 22:44:27.361629 2853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
I20260806 22:44:27.362557 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:27.364135 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:27.371038 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2853
I20260806 22:44:27.371119 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-1/wal/instance
I20260806 22:44:27.371289 747 external_mini_cluster.cc:1428] Setting key 864d754304bc34d290d47b6d3102f152
I20260806 22:44:27.372498 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.364781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.0.186.254:44613)
0806 22:44:27.364842 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.364845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.365066 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:44:27.365536 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.365537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.365548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.365601 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.366201 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.366689 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.367674 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.367881 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.367935 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.369887 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.369888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.369888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.369895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.371160 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.371593 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.371594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.371595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.371649 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.371869 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.371869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.371870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.371978 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.372038 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.372323 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.372408 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:27.372598 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.364753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44613 (local address 127.0.0.1:35788)
0806 22:44:27.364921 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.364983 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.365453 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.365670 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.365673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.365900 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.366120 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.366124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.366754 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.366755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.367600 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.367603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.367990 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.367992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.368097 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.368476 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.368483 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.369797 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.371231 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.371239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.371241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.371525 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.371705 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.371706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.371707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.371813 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.372041 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.372048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.372176 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.372290 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.372423 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260806 22:44:27.373075 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.186.252@KRBTEST.COM; defaulting to no policy
I20260806 22:44:27.379971 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.373327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34260)
0806 22:44:27.373391 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.373401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.373445 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.373659 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.373659 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.373810 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.373879 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.373881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.374684 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.374685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.375171 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.375172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.375686 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.375687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.375777 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.376112 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.376117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.377360 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.378917 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.378918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.378919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.379113 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.379369 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.379370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.379371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.379435 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.379711 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.379712 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.379756 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.379829 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.379907 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:44:27.380081 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.373371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.0.186.253:41945)
0806 22:44:27.373527 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.373531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.373549 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:44:27.373571 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.373572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.373583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.373641 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.373949 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.374599 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.375238 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.375594 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.375666 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.377458 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.377466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.377466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.377472 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.378833 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.379209 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.379212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.379212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.379305 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.379500 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.379501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.379501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.379648 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.379709 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.379856 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.379964 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 22:44:27.380643 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:27.380941 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.380843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45278)
0806 22:44:27.380881 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.380923 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 22:44:27.381069 2883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
I20260806 22:44:27.381520 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.381387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45294)
0806 22:44:27.381430 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.381470 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:44:27.381618 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38167: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
I20260806 22:44:27.397771 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:38167
--webserver_interface=127.0.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:27.428365 2844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } attempt: 3
I20260806 22:44:27.430800 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } attempt: 1
I20260806 22:44:27.431175 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.431066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45306)
0806 22:44:27.431116 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.431147 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:44:27.431593 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.431488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45310)
0806 22:44:27.431538 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.431566 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:44:27.431668 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38167: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:27.435670 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.428752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34264)
0806 22:44:27.428825 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.428838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.428896 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.429059 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.429060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.429204 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.429269 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.429271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.430181 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.430181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.430697 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.430699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.431235 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.431237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.431347 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.431800 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.431804 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.432957 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.434726 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.434727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.434727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.434888 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.435104 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.435104 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.435104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.435153 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.435438 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.435439 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.435474 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.435537 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.435607 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:44:27.435739 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.428816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.0.186.253:41945)
0806 22:44:27.428888 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.428891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.428904 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:27.428940 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.428941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.428955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.429027 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.429338 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.430106 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.430795 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.431149 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.431220 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.433036 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.433038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.433038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.433046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.434636 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.434960 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.434961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.434962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.435041 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.435203 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.435203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.435204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.435380 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.435448 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.435571 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.435657 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:44:27.436137 2844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:27.436444 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.436336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45320)
0806 22:44:27.436379 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.436414 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:44:27.436738 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.436661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45322)
0806 22:44:27.436696 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.436720 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 22:44:27.436820 2844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38167: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
I20260806 22:44:27.467998 2844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } attempt: 1
I20260806 22:44:27.468432 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.468315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45338)
0806 22:44:27.468373 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.468408 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:44:27.468827 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.468705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45354)
0806 22:44:27.468777 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.468804 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:44:27.468928 2844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38167: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
W20260806 22:44:27.480295 2972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:27.480499 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:27.480525 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:27.480559 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:27.481694 2972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:27.481731 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:27.481750 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:27.481766 2972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:27.481789 2972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:27.483309 2972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:44613,127.0.186.253:41945,127.0.186.252:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:27.483542 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:27.483736 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:44:27.488369 2972 server_base.cc:1034] running on GCE node
W20260806 22:44:27.488315 2980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.488296 2983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:27.488387 2981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:27.488682 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:27.488927 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:27.490087 2972 hybrid_clock.cc:648] HybridClock initialized: now 1786056267490074 us; error 27 us; skew 500 ppm
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:27.493155 2972 init.cc:377] Logged in from keytab as kudu/127.0.186.252@KRBTEST.COM (short username kudu)
I20260806 22:44:27.493697 2972 webserver.cc:460] Webserver started at http://127.0.186.252:35693/ using document root <none> and password file <none>
I20260806 22:44:27.493938 2972 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:27.494001 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:27.494138 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:27.495494 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data/instance:
uuid: "f90b0660a6cb4baf87c90aa2546b866f"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "40279df01be5abeee60242ab7e0f4dd3"
server_key_iv: "f4b29ca09ee06c298c4072473c4d5a3c"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.495923 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal/instance:
uuid: "f90b0660a6cb4baf87c90aa2546b866f"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "40279df01be5abeee60242ab7e0f4dd3"
server_key_iv: "f4b29ca09ee06c298c4072473c4d5a3c"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.497749 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:27.498473 2994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:27.498620 2972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:27.498706 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "f90b0660a6cb4baf87c90aa2546b866f"
format_stamp: "Formatted at 2026-08-06 22:44:27 on dist-test-slave-gjrz"
server_key: "40279df01be5abeee60242ab7e0f4dd3"
server_key_iv: "f4b29ca09ee06c298c4072473c4d5a3c"
server_key_version: "encryptionkey@0"
I20260806 22:44:27.498780 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:27.499799 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } attempt: 2
I20260806 22:44:27.500200 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.500079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45368)
0806 22:44:27.500133 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.500170 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:44:27.500499 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.500415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45384)
0806 22:44:27.500452 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.500479 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:27.500576 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38167: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38167: connect: Connection refused (error 111)
I20260806 22:44:27.507715 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:27.509656 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:27.509770 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:27.513720 2972 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.252:38167
I20260806 22:44:27.513759 3082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.252:38167 every 8 connection(s)
I20260806 22:44:27.514091 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
I20260806 22:44:27.515100 3083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:27.516659 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:27.522483 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 2972
I20260806 22:44:27.522699 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055997241161-747-0/minicluster-data/master-2/wal/instance
I20260806 22:44:27.522900 747 external_mini_cluster.cc:1428] Setting key 6a0db7da31cf81c4cc286881542567f9
I20260806 22:44:27.525254 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.517398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.0.186.254:44613)
0806 22:44:27.517474 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.517476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.517670 (+ 194us) server_negotiation.cc:407] Connection header received
0806 22:44:27.518109 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.518110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.518119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.518170 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.518763 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.519242 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.520225 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.520428 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.520487 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.522565 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.522567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.522567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.522575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.524011 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.524405 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.524406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.524406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.524462 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.524675 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.524676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.524677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.524786 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.524841 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.525088 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.525166 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:44:27.525408 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.517355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:44613 (local address 127.0.0.1:35796)
0806 22:44:27.517553 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.517588 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.518034 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.518237 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.518240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.518467 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.518666 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.518670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.519352 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.519352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.520141 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.520143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.520498 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.520499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.520611 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.521021 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.521028 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.522450 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.524118 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.524125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.524127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.524338 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.524515 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.524516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.524524 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.524619 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.524840 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.524847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.524972 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.525058 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.525205 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260806 22:44:27.525914 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:27.531078 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.523256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.0.186.252:38167)
0806 22:44:27.523412 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.523417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.523733 (+ 316us) server_negotiation.cc:407] Connection header received
0806 22:44:27.523775 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.523778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.523791 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.523854 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.524428 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.525080 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.525982 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.526361 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.526423 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.528467 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.528475 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.528476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.528482 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.530004 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.530311 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.530314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.530314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.530364 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.530528 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.530529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.530529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.530658 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.530789 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.530869 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.530941 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260806 22:44:27.532209 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.526142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:41945 (local address 127.0.0.1:34266)
0806 22:44:27.526182 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.526191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.526224 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.526376 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.526377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.526499 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.526546 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.526548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.527210 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.527211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.527724 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.527726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.528066 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.528067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.528132 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.528448 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.528452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.529736 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.531056 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.531057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.531057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.531370 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.531569 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.531570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.531570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.531626 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.531952 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.531953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.531997 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.532074 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.532152 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:44:27.532277 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.526224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.0.186.253:41945)
0806 22:44:27.526285 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.526287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.526295 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:27.526316 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.526317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.526327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.526368 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.526612 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.527141 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.527796 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.528006 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.528059 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.529811 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.529813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.529813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.529819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.530967 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.531453 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.531454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.531455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.531507 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.531711 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.531712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.531712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.531843 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.531907 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.532115 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.532188 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:44:27.532593 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:27.538774 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.531908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.0.186.254:44613)
0806 22:44:27.531942 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.531944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.531950 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:44:27.531971 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.531972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.531981 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.532035 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.532598 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.533206 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.534114 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.534408 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.534467 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.536394 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.536395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.536396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.536402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.537713 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.537999 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.538000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.538001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.538076 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.538247 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.538248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.538248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.538354 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.538502 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.538584 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.538645 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:44:27.539192 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.532806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45402)
0806 22:44:27.532876 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.532885 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.532922 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.533017 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.533018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.533123 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.533148 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.533149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.533940 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.533940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.534591 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.534594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.534970 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.534971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.535008 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.535324 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.535329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.536728 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.538178 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.538179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.538179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.538349 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.538557 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.538557 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.538558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.538620 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.538869 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.538870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.538917 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.538981 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.539070 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 22:44:27.539278 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.532844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.0.186.252:38167)
0806 22:44:27.532876 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.532877 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.532914 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:44:27.532955 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.532956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.532964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.533013 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.533197 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.533881 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.534661 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.534896 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.534971 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.536801 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.536802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.536803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.536808 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.538120 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.538437 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.538438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.538438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.538493 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.538681 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.538681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.538682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.538797 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.538866 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.539011 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.539128 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":16}
I20260806 22:44:27.540657 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: Bootstrap starting.
I20260806 22:44:27.541610 3083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:27.541931 3083 log.cc:824] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:27.542625 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: No bootstrap required, opened a new log
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:27.544066 2844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } attempt: 2
I20260806 22:44:27.544562 3083 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.544677 3083 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90b0660a6cb4baf87c90aa2546b866f, State: Initialized, Role: FOLLOWER
I20260806 22:44:27.544732 3083 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.545022 3083 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:27.545328 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.545517 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.546126 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.539376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.0.186.253:41945)
0806 22:44:27.539412 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.539414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.539422 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:27.539489 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.539490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.539499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.539549 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.540098 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.540507 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.541377 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.541563 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.541622 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.543588 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.543589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.543590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.543596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.544903 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.545255 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.545256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.545257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.545315 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.545525 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.545526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.545526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.545650 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.545805 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.545968 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.546044 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:44:27.546844 3110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:44:27.547569 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 22:44:27.546943 3110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:27.548712 2809 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "697f2ca682bd401b8dce643aec1c39fd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35800
I20260806 22:44:27.547672 2972 master_runner.cc:186] Error getting master registration for 127.0.186.254:44613: OK, instance_id { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" instance_seqno: 1786056267202612 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:44:27.550184 2972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:27.552054 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.544303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45408)
0806 22:44:27.544349 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.544358 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.544394 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.545124 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.545125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.545275 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.545312 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.545313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.546186 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.546187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.546895 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.546898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.547487 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.547487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.547541 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.547878 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.547883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.549249 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.550980 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.550981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.550981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.551188 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.551418 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.551419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.551419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.551487 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.551759 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.551760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.551805 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.551873 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.551967 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:44:27.552060 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.544344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.0.186.252:38167)
0806 22:44:27.544990 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.544992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.544998 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:44:27.545015 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.545016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.545025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.545077 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.545417 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.546084 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.547069 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.547348 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.547402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.549334 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.549335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.549336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.549343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.550894 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.551257 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.551258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.551258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.551358 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.551556 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.551556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.551557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.551697 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.551756 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.551899 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.551969 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:44:27.553467 2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: Bootstrap starting.
I20260806 22:44:27.554169 2844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:27.554461 2844 log.cc:824] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:27.555217 2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: No bootstrap required, opened a new log
I20260806 22:44:27.556514 2844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.556645 2844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f2ca682bd401b8dce643aec1c39fd, State: Initialized, Role: FOLLOWER
I20260806 22:44:27.556715 2844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.557008 2844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:27.556982 3115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.557029 2809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:44:27.557138 3115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.557197 2809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:27.557998 2809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.558235 2809 leader_election.cc:288] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [CANDIDATE]: Term 1 election: Requested vote from peers 6c541324d60447cb96d30e25c5c8bc8d (127.0.186.253:41945), f90b0660a6cb4baf87c90aa2546b866f (127.0.186.252:38167)
W20260806 22:44:27.558877 3131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:27.558923 3131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:27.559185 3045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "697f2ca682bd401b8dce643aec1c39fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f90b0660a6cb4baf87c90aa2546b866f"
I20260806 22:44:27.560276 3045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:27.561190 3045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f2ca682bd401b8dce643aec1c39fd in term 1.
I20260806 22:44:27.561406 2766 leader_election.cc:302] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f2ca682bd401b8dce643aec1c39fd, f90b0660a6cb4baf87c90aa2546b866f; no voters:
I20260806 22:44:27.561549 3115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:27.561677 3115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [term 1 LEADER]: Becoming Leader. State: Replica: 697f2ca682bd401b8dce643aec1c39fd, State: Running, Role: LEADER
I20260806 22:44:27.561760 3115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.562148 3115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697f2ca682bd401b8dce643aec1c39fd. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.562229 3115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:27.559638 2742 master_runner.cc:186] Error getting master registration for 127.0.186.253:41945: OK, instance_id { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" instance_seqno: 1786056267360295 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:44:27.562358 2742 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:27.562608 3148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:27.562824 3148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:27.571743 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.561926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.0.186.252:38167)
0806 22:44:27.562077 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.562079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.563255 (+ 1176us) server_negotiation.cc:407] Connection header received
0806 22:44:27.563506 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.563507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.563527 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.563555 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.564241 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.564889 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.565872 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.566131 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.566169 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.567592 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.567593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.567593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.567601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.569184 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.570547 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.570548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.570548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.570625 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.570978 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.570979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.570980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.571117 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.571214 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.571576 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.571654 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260806 22:44:27.572088 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.561611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.0.186.253:41945)
0806 22:44:27.561651 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.561653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.562186 (+ 533us) server_negotiation.cc:407] Connection header received
0806 22:44:27.562282 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.562283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.562294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.562322 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.563049 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.563726 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.564806 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.565139 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.565208 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.568776 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.568778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.568778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.568785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.570339 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.570824 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.570825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.570826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.570886 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.571130 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.571131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.571132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.571303 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.571377 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.571910 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.571993 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:44:27.575799 3045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [term 1 FOLLOWER]: Refusing update from remote peer 697f2ca682bd401b8dce643aec1c39fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:27.576164 3142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:27.577512 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697f2ca682bd401b8dce643aec1c39fd. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.577595 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.578685 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.563178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.0.186.254:44613)
0806 22:44:27.563218 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.563220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.564884 (+ 1664us) server_negotiation.cc:407] Connection header received
0806 22:44:27.564974 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.564975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.564984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.565012 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.565661 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.566360 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.573612 (+ 7252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.573914 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.573976 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.575318 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.575319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.575319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.575339 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.577019 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.577584 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.577585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.577586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.577652 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.577917 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.577918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.577918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.578040 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.578119 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.578524 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.578599 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:44:27.588339 3115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.588440 3115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:27.588501 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.588578 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.589200 3153 mvcc.cc:205] Tried to move back new op lower bound from 7315686471982624768 to 7315686471933665280. Current Snapshot: MvccSnapshot[applied={T|T < 7315686471982624768}]
I20260806 22:44:27.591652 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } attempt: 3
Aug 06 22:44:27 dist-test-slave-gjrz krb5kdc[2719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:27.600281 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.591921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38167 (local address 127.0.0.1:45434)
0806 22:44:27.591958 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:27.591969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:27.592008 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:27.592162 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:27.592163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:27.592324 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:27.592380 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.592382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.593244 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.593245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.594031 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:27.594033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.594484 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.594484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:27.594540 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.595048 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:27.595053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:27.596535 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:27.598950 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.598952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.598952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.599173 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.599516 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:27.599517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:27.599518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:27.599577 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:27.599974 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:27.599975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:27.600032 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 22:44:27.600077 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:44:27.600175 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:44:27.601039 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:27.591975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.0.186.252:38167)
0806 22:44:27.592026 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.592028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.592038 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:27.592090 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.592091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.592100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.592148 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.592453 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.593166 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.594112 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:27.594407 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:27.594482 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:27.596621 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:27.596622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:27.596623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:27.596630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:27.598847 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.599324 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.599325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.599325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.599405 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:27.599688 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:27.599689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:27.599689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:27.599879 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:27.600708 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:27.600772 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:44:27.600894 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:27.602865 2961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: Bootstrap starting.
I20260806 22:44:27.603641 2961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:27.603808 3148 catalog_manager.cc:1347] Generated new cluster ID: 6578e6a70bef467dbad4284b55fe5a5e
I20260806 22:44:27.603856 3148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:27.603952 2961 log.cc:824] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:27.608071 2961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: No bootstrap required, opened a new log
I20260806 22:44:27.610085 2961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.610214 2961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c541324d60447cb96d30e25c5c8bc8d, State: Initialized, Role: FOLLOWER
I20260806 22:44:27.610285 2961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } }
I20260806 22:44:27.610605 2961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:27.611430 2916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "697f2ca682bd401b8dce643aec1c39fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6c541324d60447cb96d30e25c5c8bc8d"
I20260806 22:44:27.611541 2916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:27.612376 2916 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f2ca682bd401b8dce643aec1c39fd in term 1.
I20260806 22:44:27.612840 2915 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [term 1 FOLLOWER]: Refusing update from remote peer 697f2ca682bd401b8dce643aec1c39fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:27.613112 3142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:27.615460 3163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.615576 3163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.615634 3163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697f2ca682bd401b8dce643aec1c39fd. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.615685 3163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.615717 3163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.615760 3163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:27.620486 3174 mvcc.cc:205] Tried to move back new op lower bound from 7315686471982624768 to 7315686471933665280. Current Snapshot: MvccSnapshot[applied={T|T < 7315686471982624768}]
W20260806 22:44:27.621236 3182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:27.621331 3182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:27.635368 3142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "697f2ca682bd401b8dce643aec1c39fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "697f2ca682bd401b8dce643aec1c39fd" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 44613 } } peers { permanent_uuid: "6c541324d60447cb96d30e25c5c8bc8d" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 41945 } } peers { permanent_uuid: "f90b0660a6cb4baf87c90aa2546b866f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38167 } } }
I20260806 22:44:27.635476 3142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:27.637778 3148 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:27.638584 3148 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:27.645366 3148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 697f2ca682bd401b8dce643aec1c39fd: Generated new TSK 0
I20260806 22:44:27.645543 3148 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:44:27.672676 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.670965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.0.186.253:41945)
0806 22:44:27.671009 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.671011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.672009 (+ 998us) server_negotiation.cc:407] Connection header received
0806 22:44:27.672117 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.672118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.672129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.672159 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.672610 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34290: BlockingRecv error: recv got EOF from 127.0.0.1:34290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:44:27.673454 3087 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.671346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.0.186.252:38167)
0806 22:44:27.671385 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.671387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.672955 (+ 1568us) server_negotiation.cc:407] Connection header received
0806 22:44:27.673050 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.673050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.673060 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.673089 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.673397 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45438: BlockingRecv error: recv got EOF from 127.0.0.1:45438 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:44:27.674194 2847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.671700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.0.186.254:44613)
0806 22:44:27.671745 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.671747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.673693 (+ 1946us) server_negotiation.cc:407] Connection header received
0806 22:44:27.673790 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.673791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.673801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.673829 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.674132 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35812: BlockingRecv error: recv got EOF from 127.0.0.1:35812 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:44:27.676542 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.675788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.0.186.253:41945)
0806 22:44:27.675823 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.675825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.676019 (+ 194us) server_negotiation.cc:407] Connection header received
0806 22:44:27.676120 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.676121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.676139 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.676167 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.676484 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34306: BlockingRecv error: recv got EOF from 127.0.0.1:34306 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:44:27.679565 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.678744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.0.186.253:41945)
0806 22:44:27.678778 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.678780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.679018 (+ 238us) server_negotiation.cc:407] Connection header received
0806 22:44:27.679112 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.679113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.679122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.679150 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.679522 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34312: BlockingRecv error: recv got EOF from 127.0.0.1:34312 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:44:27.683405 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.682616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.0.186.253:41945)
0806 22:44:27.682649 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.682651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.682900 (+ 249us) server_negotiation.cc:407] Connection header received
0806 22:44:27.682994 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.682995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.683007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.683035 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.683363 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34320: BlockingRecv error: recv got EOF from 127.0.0.1:34320 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:44:27.689194 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.688421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34324 (local address 127.0.186.253:41945)
0806 22:44:27.688456 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.688457 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.688712 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:44:27.688806 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.688807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.688816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.688844 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.689153 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34324: BlockingRecv error: recv got EOF from 127.0.0.1:34324 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:44:27.699033 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.698246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.0.186.253:41945)
0806 22:44:27.698278 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.698280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.698534 (+ 254us) server_negotiation.cc:407] Connection header received
0806 22:44:27.698628 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.698629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.698640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.698668 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.698974 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34328: BlockingRecv error: recv got EOF from 127.0.0.1:34328 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 22:44:27.717133 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.716336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.0.186.253:41945)
0806 22:44:27.716372 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.716374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.716623 (+ 249us) server_negotiation.cc:407] Connection header received
0806 22:44:27.716717 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.716718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.716729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.716755 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.717084 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34334: BlockingRecv error: recv got EOF from 127.0.0.1:34334 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:44:27.750957 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.750278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.0.186.253:41945)
0806 22:44:27.750333 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.750335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.750464 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:44:27.750525 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.750526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.750537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.750579 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.750904 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34350: BlockingRecv error: recv got EOF from 127.0.0.1:34350 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:44:27.816941 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.816201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.0.186.253:41945)
0806 22:44:27.816267 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.816269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.816375 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:44:27.816462 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.816463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.816476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.816524 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.816869 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34354: BlockingRecv error: recv got EOF from 127.0.0.1:34354 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:44:27.946813 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:27.946128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.0.186.253:41945)
0806 22:44:27.946234 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:27.946236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:27.946252 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:44:27.946336 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:27.946337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:27.946348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:27.946399 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:27.946747 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34362: BlockingRecv error: recv got EOF from 127.0.0.1:34362 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:44:28.204574 2967 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:28.203986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.0.186.253:41945)
0806 22:44:28.204042 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.204044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.204074 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:44:28.204099 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.204099 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.204110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.204148 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:28.204518 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34368: BlockingRecv error: recv got EOF from 127.0.0.1:34368 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:44:28.548058 3110 catalog_manager.cc:1259] Loaded cluster ID: 6578e6a70bef467dbad4284b55fe5a5e
I20260806 22:44:28.548134 3110 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: loading cluster ID for follower catalog manager: success
I20260806 22:44:28.548971 3110 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: acquiring CA information for follower catalog manager: success
I20260806 22:44:28.549283 3110 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f90b0660a6cb4baf87c90aa2546b866f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:28.621810 3182 catalog_manager.cc:1259] Loaded cluster ID: 6578e6a70bef467dbad4284b55fe5a5e
I20260806 22:44:28.621902 3182 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: loading cluster ID for follower catalog manager: success
I20260806 22:44:28.622747 3182 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: acquiring CA information for follower catalog manager: success
I20260806 22:44:28.623073 3182 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6c541324d60447cb96d30e25c5c8bc8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:28.719985 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:28.717648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.0.186.253:41945)
0806 22:44:28.717758 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.717760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.717799 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:44:28.717827 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.717828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.717880 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.717966 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:28.718090 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.718580 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.719315 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.719525 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.719577 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:28.719710 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:28.719824 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:28.719879 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:28.719892 (+ 13us) server_negotiation.cc:299] Negotiation successful
0806 22:44:28.719909 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260806 22:44:28.723963 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:28.721705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.0.186.252:38167)
0806 22:44:28.721820 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.721822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.721829 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:28.721881 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.721882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.721918 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.721984 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:28.722093 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.722549 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.723216 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.723446 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.723500 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:28.723644 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:28.723761 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:28.723821 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:28.723864 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:44:28.723884 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":50,"threads_started":1}
I20260806 22:44:28.728046 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:28.725355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.0.186.253:41945)
0806 22:44:28.725395 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.725397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.725510 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:44:28.725595 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.725596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.725605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.725648 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:28.725796 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.726327 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727225 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.727551 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727608 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:28.727736 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:28.727767 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:28.727808 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:28.727955 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:44:28.727978 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:44:28.728410 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:28.725575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.0.186.252:38167)
0806 22:44:28.725619 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.725621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.725686 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:44:28.725740 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.725741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.725751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.725799 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:28.725923 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.726620 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727656 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.727896 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727960 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:28.728100 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:28.728129 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:28.728171 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:28.728323 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:44:28.728344 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:28.728540 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:28.725369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35826 (local address 127.0.186.254:44613)
0806 22:44:28.725624 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:28.725626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:28.725634 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:28.725755 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:28.725757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:28.725810 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:28.725888 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:28.725995 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.726728 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727474 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:28.727750 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:28.727811 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:28.727949 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:28.728044 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:28.728096 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:28.728445 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 22:44:28.728469 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":51,"threads_started":1}
I20260806 22:44:28.729974 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2742
I20260806 22:44:28.741324 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2853
I20260806 22:44:28.754442 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 2972
2026-08-06T22:44:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2927 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:28 dist-test-slave-gjrz krb5kdc[3312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:28 dist-test-slave-gjrz krb5kdc[3312](info): set up 2 sockets
Aug 06 22:44:28 dist-test-slave-gjrz krb5kdc[3312](info): commencing operation
krb5kdc: starting...
W20260806 22:44:29.872993 747 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/test-user.keytab.
Aug 06 22:44:29 dist-test-slave-gjrz krb5kdc[3312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
Entry for principal kudu/127.0.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab.
I20260806 22:44:29.986689 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46763
--webserver_interface=127.0.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:30.066287 3330 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:30.066505 3330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:30.066530 3330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:30.066556 3330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:30.067677 3330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:30.067715 3330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:30.067734 3330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:30.067750 3330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:30.067765 3330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:30.069223 3330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.254.keytab
--principal=kudu/127.0.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.254:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:30.069463 3330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:30.069664 3330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:30.074249 3335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.074249 3339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.074237 3336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:30.074375 3330 server_base.cc:1034] running on GCE node
I20260806 22:44:30.074680 3330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:30.074942 3330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:30.076098 3330 hybrid_clock.cc:648] HybridClock initialized: now 1786056270076084 us; error 28 us; skew 500 ppm
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:30.078608 3330 init.cc:377] Logged in from keytab as kudu/127.0.186.254@KRBTEST.COM (short username kudu)
I20260806 22:44:30.079044 3330 webserver.cc:460] Webserver started at http://127.0.186.254:36553/ using document root <none> and password file <none>
I20260806 22:44:30.079241 3330 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:30.079320 3330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:30.079454 3330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:30.080469 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data/instance:
uuid: "5344751f9ea64a2daff53956e26f0682"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "172960eb4f7cc18ad723bf53b80d8df1"
server_key_iv: "070a7b10cf623d3e9e1fd565ee349a74"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.080817 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance:
uuid: "5344751f9ea64a2daff53956e26f0682"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "172960eb4f7cc18ad723bf53b80d8df1"
server_key_iv: "070a7b10cf623d3e9e1fd565ee349a74"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.082082 3330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:30.082799 3353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:30.083002 3330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:30.083096 3330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal
uuid: "5344751f9ea64a2daff53956e26f0682"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "172960eb4f7cc18ad723bf53b80d8df1"
server_key_iv: "070a7b10cf623d3e9e1fd565ee349a74"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.083158 3330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:30.099815 3330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:30.102361 3330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:30.102502 3330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:30.106395 3330 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.254:46763
I20260806 22:44:30.106441 3432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.254:46763 every 8 connection(s)
I20260806 22:44:30.106748 3330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:30.107767 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:30.109253 3434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:30.111105 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3330
I20260806 22:44:30.111189 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-0/wal/instance
I20260806 22:44:30.111390 747 external_mini_cluster.cc:1428] Setting key 3d034ac16556eba0fd0995799227a7db
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:30.118829 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.109900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.0.186.254:46763)
0806 22:44:30.110062 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.110070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.110130 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:44:30.110675 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.110677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.110694 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.110772 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.111335 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.112186 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.113037 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.113340 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.113438 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.115801 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.115808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.115809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.115815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.117306 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.117752 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.117755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.117755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.117837 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.118070 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.118071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.118071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.118268 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.118327 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.118551 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.118674 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260806 22:44:30.118858 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.109857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46763 (local address 127.0.0.1:50770)
0806 22:44:30.110029 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.110066 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.110545 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.110788 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.110790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.111031 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.111215 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.111222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.112278 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.112279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.112956 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.112958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.113438 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.113439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.113504 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.114019 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.114028 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.115707 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.117397 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.117405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.117407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.117668 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.117901 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.117901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.117902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.118010 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.118333 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.118340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.118423 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.118527 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.118752 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":68,"thread_start_us":71,"threads_started":1}
I20260806 22:44:30.119822 3434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.186.253@KRBTEST.COM; defaulting to no policy
I20260806 22:44:30.120208 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.120073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36656)
0806 22:44:30.120124 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.120180 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:44:30.120334 3355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
I20260806 22:44:30.120812 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.120719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36660)
0806 22:44:30.120769 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.120794 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:44:30.120926 3434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:42237: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
Entry for principal kudu/127.0.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab.
I20260806 22:44:30.137395 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:42237
--webserver_interface=127.0.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:30.170147 3434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } attempt: 1
I20260806 22:44:30.170650 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.170490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36666)
0806 22:44:30.170556 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.170609 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:44:30.171005 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.170912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36672)
0806 22:44:30.170957 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.170985 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:44:30.171118 3434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:42237: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
W20260806 22:44:30.217748 3448 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:30.217940 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:30.217955 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:30.217983 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:30.219048 3448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:30.219079 3448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:30.219090 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:30.219101 3448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:30.219112 3448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:30.220626 3448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.253.keytab
--principal=kudu/127.0.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.253:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:30.220815 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:30.220994 3448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:30.225419 3462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.225436 3464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.225437 3460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:30.225634 3448 server_base.cc:1034] running on GCE node
I20260806 22:44:30.225821 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:30.226059 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:30.227202 3448 hybrid_clock.cc:648] HybridClock initialized: now 1786056270227194 us; error 28 us; skew 500 ppm
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:30.229504 3448 init.cc:377] Logged in from keytab as kudu/127.0.186.253@KRBTEST.COM (short username kudu)
I20260806 22:44:30.229892 3448 webserver.cc:460] Webserver started at http://127.0.186.253:36613/ using document root <none> and password file <none>
I20260806 22:44:30.230104 3448 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:30.230156 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:30.230278 3448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:30.231313 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data/instance:
uuid: "ffc3e4c18a6d423abdeb073706353749"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "65de42f127cec338b534896e3c0b0712"
server_key_iv: "86566df33d18809144b709afed72a140"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.231653 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal/instance:
uuid: "ffc3e4c18a6d423abdeb073706353749"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "65de42f127cec338b534896e3c0b0712"
server_key_iv: "86566df33d18809144b709afed72a140"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.232836 3448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:30.233466 3473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:30.233613 3448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:30.233704 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal
uuid: "ffc3e4c18a6d423abdeb073706353749"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "65de42f127cec338b534896e3c0b0712"
server_key_iv: "86566df33d18809144b709afed72a140"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.233770 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:30.239254 3434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } attempt: 2
I20260806 22:44:30.239650 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.239540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36688)
0806 22:44:30.239588 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.239622 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:44:30.239984 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.239886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36692)
0806 22:44:30.239937 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.239962 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:44:30.240064 3434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.253:42237: Network error: Client connection negotiation failed: client connection to 127.0.186.253:42237: connect: Connection refused (error 111)
I20260806 22:44:30.271924 3448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:30.273797 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:30.273941 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:30.277788 3448 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.253:42237
I20260806 22:44:30.277833 3558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.253:42237 every 8 connection(s)
I20260806 22:44:30.278148 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/data/info.pb
I20260806 22:44:30.279165 3560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:30.280738 3560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:30.282254 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3448
I20260806 22:44:30.282343 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-1/wal/instance
I20260806 22:44:30.282534 747 external_mini_cluster.cc:1428] Setting key 4ff468db0de4e9129f1ea34416212d38
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:30.289788 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.281390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.0.186.254:46763)
0806 22:44:30.281460 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.281462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.281593 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:44:30.282081 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.282083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.282100 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.282173 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.282797 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.283478 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.284477 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.284781 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.284853 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.286981 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.286982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.286983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.286990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.288512 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.288900 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.288901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.288901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.288966 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.289178 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.289179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.289179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.289320 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.289389 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.289614 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.289697 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:44:30.289847 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.281338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46763 (local address 127.0.0.1:50780)
0806 22:44:30.281483 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.281519 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.281999 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.282226 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.282229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.282500 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.282710 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.282716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.283569 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.283570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.284385 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.284389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.284878 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.284879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.285012 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.285504 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.285513 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.286891 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.288608 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.288615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.288616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.288831 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.289027 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.289028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.289028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.289122 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.289377 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.289381 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.289497 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.289575 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.289691 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260806 22:44:30.290405 3560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.186.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.186.252@KRBTEST.COM" created.
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:30.297646 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.290715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.0.186.253:42237)
0806 22:44:30.290843 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.290847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.290863 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:44:30.290882 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.290893 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.290902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.290961 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.291216 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.291903 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.292538 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.292836 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.292902 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.294702 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.294711 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.294711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.294717 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.296376 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.296714 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.296717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.296717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.296778 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.296962 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.296963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.296963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.297158 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.297253 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.297398 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.297503 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260806 22:44:30.297688 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.290650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36700)
0806 22:44:30.290690 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.290700 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.290738 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.290972 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.290973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.291106 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.291152 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.291153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.291981 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.291982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.292472 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.292474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.292900 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.292901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.292961 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.293297 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.293302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.294625 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.296471 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.296472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.296472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.296634 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.296834 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.296835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.296835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.296893 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.297252 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.297253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.297293 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.297359 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.297568 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":93}
I20260806 22:44:30.298406 3560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:30.298728 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.298599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55856)
0806 22:44:30.298644 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.298699 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:30.298862 3477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
I20260806 22:44:30.299301 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.299168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55866)
0806 22:44:30.299215 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.299278 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:44:30.299423 3560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38351: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
Entry for principal kudu/127.0.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab.
I20260806 22:44:30.309824 747 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwWuo9B/build/release/bin/kudu
/tmp/dist-test-taskwWuo9B/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:38351
--webserver_interface=127.0.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.186.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:44:30.331233 3434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } attempt: 3
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:30.338400 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.331629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36714)
0806 22:44:30.331699 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.331712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.331773 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.331931 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.331932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.332076 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.332159 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.332160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.333070 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.333070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.333606 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.333608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.334101 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.334102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.334184 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.334634 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.334639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.335939 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.337411 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.337412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.337413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.337610 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.337794 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.337795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.337795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.337854 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.338079 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.338080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.338128 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.338186 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.338295 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:44:30.338400 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.331694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.0.186.253:42237)
0806 22:44:30.331759 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.331761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.331774 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:30.331824 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.331826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.331839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.331899 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.332234 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.332996 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.333674 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.334009 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.334083 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.336034 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.336036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.336036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.336043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.337316 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.337683 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.337684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.337685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.337739 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.337911 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.337912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.337912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.338023 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.338080 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.338215 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.338297 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:44:30.338773 3434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:30.339121 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.339027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55870)
0806 22:44:30.339071 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.339100 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:44:30.339439 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.339352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55884)
0806 22:44:30.339391 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.339416 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:44:30.339552 3434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38351: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
I20260806 22:44:30.348636 3560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } attempt: 1
I20260806 22:44:30.349062 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.348935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55886)
0806 22:44:30.348991 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.349031 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:44:30.349355 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.349272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55888)
0806 22:44:30.349311 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.349336 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:44:30.349444 3560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38351: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
I20260806 22:44:30.370697 3434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } attempt: 1
I20260806 22:44:30.371132 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.371003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55892)
0806 22:44:30.371067 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.371105 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 22:44:30.371488 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.371391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55906)
0806 22:44:30.371438 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.371465 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:44:30.371626 3434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38351: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
W20260806 22:44:30.392261 3569 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:44:30.392452 3569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:30.392475 3569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:30.392501 3569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:30.393602 3569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:30.393640 3569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:30.393659 3569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:30.393674 3569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:30.393692 3569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:30.395241 3569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.186.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.186.254:46763,127.0.186.253:42237,127.0.186.252:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/krb5kdc/kudu_127.0.186.252.keytab
--principal=kudu/127.0.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.186.252:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:30.395533 3569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:30.395736 3569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:30.400142 3579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.400151 3577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:30.400396 3581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:30.400497 3569 server_base.cc:1034] running on GCE node
I20260806 22:44:30.400695 3569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:30.400921 3569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:30.402053 3569 hybrid_clock.cc:648] HybridClock initialized: now 1786056270402041 us; error 27 us; skew 500 ppm
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:30.404559 3569 init.cc:377] Logged in from keytab as kudu/127.0.186.252@KRBTEST.COM (short username kudu)
I20260806 22:44:30.404959 3569 webserver.cc:460] Webserver started at http://127.0.186.252:44071/ using document root <none> and password file <none>
I20260806 22:44:30.405176 3569 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:30.405223 3569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:30.405330 3569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:30.406358 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data/instance:
uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "2e7339289911e93d9ba1ec8ba123e4d4"
server_key_iv: "501f48cdc726314cf78d62c8ef8c5e32"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.406692 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal/instance:
uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "2e7339289911e93d9ba1ec8ba123e4d4"
server_key_iv: "501f48cdc726314cf78d62c8ef8c5e32"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.407986 3569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:30.408644 3591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:30.408787 3569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:30.408864 3569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data,/tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal
uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f"
format_stamp: "Formatted at 2026-08-06 22:44:30 on dist-test-slave-gjrz"
server_key: "2e7339289911e93d9ba1ec8ba123e4d4"
server_key_iv: "501f48cdc726314cf78d62c8ef8c5e32"
server_key_version: "encryptionkey@0"
I20260806 22:44:30.408927 3569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:30.417487 3569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:30.417599 3560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } attempt: 2
I20260806 22:44:30.417961 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.417837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55914)
0806 22:44:30.417885 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.417923 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 22:44:30.418309 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.418214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55920)
0806 22:44:30.418257 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.418283 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:44:30.418437 3560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.186.252:38351: Network error: Client connection negotiation failed: client connection to 127.0.186.252:38351: connect: Connection refused (error 111)
I20260806 22:44:30.419476 3569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:30.419610 3569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:30.423377 3569 rpc_server.cc:307] RPC server started. Bound to: 127.0.186.252:38351
I20260806 22:44:30.423415 3692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.186.252:38351 every 8 connection(s)
I20260806 22:44:30.423717 3569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/data/info.pb
I20260806 22:44:30.424640 3694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:30.424688 747 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwWuo9B/build/release/bin/kudu as pid 3569
I20260806 22:44:30.424798 747 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwWuo9B/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055997241161-747-0/minicluster-data/master-2/wal/instance
I20260806 22:44:30.425009 747 external_mini_cluster.cc:1428] Setting key 04591302b33bc317b18bc6a18b09cefe
I20260806 22:44:30.426656 3694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:30.434384 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.427228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.0.186.254:46763)
0806 22:44:30.427295 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.427297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.427460 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:44:30.427538 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.427539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.427549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.427597 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.428028 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.428494 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.429244 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.429451 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.429524 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.431643 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.431645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.431645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.431653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.433055 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.433508 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.433508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.433509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.433554 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.433796 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.433797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.433797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.433924 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.433985 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.434258 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.434322 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:44:30.434618 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.427194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.254:46763 (local address 127.0.0.1:50782)
0806 22:44:30.427370 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.427400 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.427475 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.427623 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.427626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.427858 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.427942 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.427947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.428574 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.428574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.429165 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.429167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.429588 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.429590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.429670 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.430006 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.430016 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.431551 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.433132 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.433138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.433140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.433428 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.433607 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.433608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.433608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.433737 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.433980 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.433983 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.434135 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.434232 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.434382 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260806 22:44:30.435141 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.425466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.0.186.252:38351)
0806 22:44:30.425639 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.425649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.425675 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:44:30.426094 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.426100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.426144 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.426261 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.426873 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.427949 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.428696 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.429099 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.429198 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.432054 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.432063 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.432063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.432069 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.433628 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.433978 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.433988 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.433988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.434059 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.434223 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.434224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.434224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.434377 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.434885 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.434989 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.435068 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260806 22:44:30.435144 3694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.254@KRBTEST.COM
I20260806 22:44:30.441543 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.435584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.253:42237 (local address 127.0.0.1:36716)
0806 22:44:30.435678 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.435686 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.435732 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.435912 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.435913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.436042 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.436091 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.436092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.436730 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.436731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.437240 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.437241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.437621 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.437622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.437679 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.437998 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.438002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.439105 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.440691 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.440692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.440692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.440839 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.441028 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.441029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.441029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.441076 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.441285 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.441285 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.441328 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.441382 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.441461 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 22:44:30.441565 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.435640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.0.186.253:42237)
0806 22:44:30.435691 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.435692 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.435719 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:44:30.435830 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.435831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.435856 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.435908 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.436159 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.436658 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.437321 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.437529 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.437583 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.439186 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.439187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.439188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.439194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.440620 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.440905 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.440907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.440907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.440964 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.441122 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.441123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.441123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.441223 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.441272 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.441420 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.441494 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:44:30.442060 3694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:44:30.442510 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.435841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.0.186.254:46763)
0806 22:44:30.435882 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.435884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.435892 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:30.435929 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.435931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.435939 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.435989 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.436404 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.436879 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.437543 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.437733 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.437786 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.439841 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.439842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.439843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.439850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.441021 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.441327 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.441328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.441328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.441377 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.441558 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.441559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.441559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.442109 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.442312 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.442351 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.442415 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.252@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.186.253@KRBTEST.COM
I20260806 22:44:30.446748 3434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } attempt: 2
I20260806 22:44:30.448549 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.442454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55948)
0806 22:44:30.442545 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.442553 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.442611 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.442720 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.442720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.442850 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.442881 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.442883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.443650 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.443651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.444098 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.444099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.444407 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.444407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.444451 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.444764 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.444768 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.445979 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.447658 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.447659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.447659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.447853 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.448026 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.448027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.448027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.448070 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.448275 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.448276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.448310 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.448370 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.448462 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 22:44:30.448791 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.442516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.0.186.252:38351)
0806 22:44:30.442555 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.442557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.442584 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:44:30.442656 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.442657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.442665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.442709 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.442959 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.443576 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.444162 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.444353 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.444403 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.446056 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.446057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.446057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.446063 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.447582 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.447921 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.447922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.447922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.447969 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.448125 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.448126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.448126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.448222 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.448275 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.448395 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.448512 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":20}
I20260806 22:44:30.449739 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.443194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.0.186.253:42237)
0806 22:44:30.443235 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.443236 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.443244 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:30.443288 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.443289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.443298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.443340 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.443744 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.444163 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.444748 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.444941 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.444992 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.447129 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.447131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.447131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.447137 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.448667 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.449041 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.449042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.449043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.449094 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.449259 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.449260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.449260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.449389 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.449552 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.449586 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.449664 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:44:30.450932 3694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: Bootstrap starting.
I20260806 22:44:30.451502 747 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:30.451570 3694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: Neither blocks nor log segments found. Creating new log.
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.254@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:30.451864 3694 log.cc:824] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:30.452626 3694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: No bootstrap required, opened a new log
I20260806 22:44:30.452737 3390 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5344751f9ea64a2daff53956e26f0682"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50796
I20260806 22:44:30.454455 3694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.454630 3694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7d90abc7bf4b90bf4ed6b9ab386b6f, State: Initialized, Role: FOLLOWER
I20260806 22:44:30.454663 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.446976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55964)
0806 22:44:30.447132 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.447144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.447194 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.447514 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.447515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.447643 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.447681 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.447683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.449041 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.449042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.449708 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.449710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.450339 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.450340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.450399 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.450743 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.450748 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.452004 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.453663 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.453664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.453665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.453871 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.454067 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.454067 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.454068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.454127 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.454392 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.454393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.454433 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.454492 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.454573 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 22:44:30.454735 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.447015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.0.186.252:38351)
0806 22:44:30.447144 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.447146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.447179 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:44:30.447244 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.447245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.447255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.447332 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.447759 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.448942 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.449784 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.450270 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.450328 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.452108 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.452109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.452110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.452117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.453585 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.453939 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.453940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.453940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.454007 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.454179 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.454180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.454180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.454332 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.454397 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.454530 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.454604 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260806 22:44:30.454726 3694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.455926 3434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: Bootstrap starting.
I20260806 22:44:30.456496 3434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:30.456755 3434 log.cc:824] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:30.455042 3702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.455049 3694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:30.457042 3702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.457499 3434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: No bootstrap required, opened a new log
I20260806 22:44:30.458567 3434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.458716 3434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5344751f9ea64a2daff53956e26f0682, State: Initialized, Role: FOLLOWER
I20260806 22:44:30.458796 3434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.459088 3434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:30.459146 3390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
W20260806 22:44:30.459172 3718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:44:30.459194 3390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:44:30.459210 3718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:30.459988 3390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.460198 3720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.460259 3390 leader_election.cc:288] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [CANDIDATE]: Term 1 election: Requested vote from peers ffc3e4c18a6d423abdeb073706353749 (127.0.186.253:42237), 2b7d90abc7bf4b90bf4ed6b9ab386b6f (127.0.186.252:38351)
I20260806 22:44:30.460297 3720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.460765 3634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5344751f9ea64a2daff53956e26f0682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f"
I20260806 22:44:30.460870 3634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:30.459059 3569 master_runner.cc:186] Error getting master registration for 127.0.186.254:46763: OK, instance_id { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" instance_seqno: 1786056270105344 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260806 22:44:30.460981 3734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:30.461018 3734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:30.461054 3569 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:30.461678 3634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5344751f9ea64a2daff53956e26f0682 in term 1.
I20260806 22:44:30.462067 3357 leader_election.cc:302] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7d90abc7bf4b90bf4ed6b9ab386b6f, 5344751f9ea64a2daff53956e26f0682; no voters:
I20260806 22:44:30.462330 3720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:30.462594 3720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [term 1 LEADER]: Becoming Leader. State: Replica: 5344751f9ea64a2daff53956e26f0682, State: Running, Role: LEADER
I20260806 22:44:30.462678 3720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.462812 3330 master_runner.cc:186] Error getting master registration for 127.0.186.253:42237: OK, instance_id { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" instance_seqno: 1786056270276747 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:44:30.464968 3330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:44:30.465706 3748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5344751f9ea64a2daff53956e26f0682. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.465826 3748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:30.466218 3755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:30.466423 3755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:30.469228 3634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [term 1 FOLLOWER]: Refusing update from remote peer 5344751f9ea64a2daff53956e26f0682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:30.470357 3720 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:30.473090 3702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5344751f9ea64a2daff53956e26f0682. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.473178 3702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.474546 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.461708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36736 (local address 127.0.186.253:42237)
0806 22:44:30.461850 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.461852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.466066 (+ 4214us) server_negotiation.cc:407] Connection header received
0806 22:44:30.467152 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.467153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.467164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.467191 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.467901 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.468595 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.470009 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.470295 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.470340 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.471732 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.471733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.471733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.471739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.473301 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.473727 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.473729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.473729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.473786 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.473981 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.473982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.473982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.474109 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.474185 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.474370 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.474447 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 22:44:30.474875 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.461801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50808 (local address 127.0.186.254:46763)
0806 22:44:30.461899 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.461901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.465290 (+ 3389us) server_negotiation.cc:407] Connection header received
0806 22:44:30.465420 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.465421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.465444 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.465474 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.466752 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.467437 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.468918 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.469229 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.469267 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.471648 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.471662 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.471663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.471669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.473194 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.473838 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.473839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.473839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.473901 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.474197 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.474198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.474198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.474332 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.474406 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.474726 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.474793 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:44:30.477954 3748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.478053 3748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:30.478125 3702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.478206 3702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.478552 3757 mvcc.cc:205] Tried to move back new op lower bound from 7315686483839897600 to 7315686483825758208. Current Snapshot: MvccSnapshot[applied={T|T < 7315686483839897600}]
I20260806 22:44:30.480185 3755 catalog_manager.cc:1347] Generated new cluster ID: bc1ba5fae59d4aa6be0bbafa2ba39cd9
I20260806 22:44:30.480233 3755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:30.486603 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.462271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.0.186.252:38351)
0806 22:44:30.462304 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.462305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.470725 (+ 8420us) server_negotiation.cc:407] Connection header received
0806 22:44:30.470847 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.470848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.470857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.470885 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.475046 (+ 4161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.475778 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.476726 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.477031 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.477094 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.478900 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.478902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.478902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.478910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.480492 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.485677 (+ 5185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.485678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.485678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.485731 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.485900 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.485901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.485901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.486020 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.486110 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.486443 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.486521 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:44:30.503324 3755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:30.504032 3755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:30.509580 3560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } attempt: 3
Aug 06 22:44:30 dist-test-slave-gjrz krb5kdc[3312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.186.253@KRBTEST.COM for kudu/127.0.186.252@KRBTEST.COM
I20260806 22:44:30.516825 3755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682: Generated new TSK 0
I20260806 22:44:30.517314 3755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:30.519083 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.509821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.186.252:38351 (local address 127.0.0.1:55978)
0806 22:44:30.509895 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:30.509905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:30.509937 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:30.510165 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:30.510167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:30.510341 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:30.510385 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.510386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.511355 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.511356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.512134 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:30.512136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.512894 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.512895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:30.512966 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.513397 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:30.513402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:30.514825 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:30.517986 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.517987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.517987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.518204 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.518375 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:30.518376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:30.518376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:30.518451 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:30.518786 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:30.518787 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:30.518831 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:44:30.518892 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:30.518990 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 22:44:30.519203 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:30.510004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.0.186.252:38351)
0806 22:44:30.510049 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.510050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.510056 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:44:30.510073 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.510074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.510093 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.510447 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.510501 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.511242 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.512300 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:30.512796 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:30.513164 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:30.514957 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:30.514958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:30.514958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:30.514969 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:30.517912 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.518256 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.518257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.518257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.518328 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:30.518537 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:30.518538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:30.518538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:30.518707 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:30.518806 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:30.518951 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:44:30.519072 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:44:30.521059 3560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: Bootstrap starting.
I20260806 22:44:30.521788 3560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:30.522125 3560 log.cc:824] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:30.522994 3560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: No bootstrap required, opened a new log
I20260806 22:44:30.524927 3560 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.525072 3560 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc3e4c18a6d423abdeb073706353749, State: Initialized, Role: FOLLOWER
I20260806 22:44:30.525154 3560 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } }
I20260806 22:44:30.525491 3560 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:30.525620 3516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:30.526532 3516 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [term 1 FOLLOWER]: Refusing update from remote peer 5344751f9ea64a2daff53956e26f0682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:30.526753 3517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5344751f9ea64a2daff53956e26f0682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ffc3e4c18a6d423abdeb073706353749"
I20260806 22:44:30.526906 3763 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:30.525434 3766 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.527308 3766 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.527580 3517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5344751f9ea64a2daff53956e26f0682 in term 1.
W20260806 22:44:30.528396 3785 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:30.530130 3785 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:30.530481 3766 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5344751f9ea64a2daff53956e26f0682. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.530545 3766 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.530576 3766 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:30.530611 3766 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:44:30.535907 3788 mvcc.cc:205] Tried to move back new op lower bound from 7315686484023341056 to 7315686483825758208. Current Snapshot: MvccSnapshot[applied={T|T < 7315686483839897600}]
W20260806 22:44:30.556175 3700 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.554466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.0.186.252:38351)
0806 22:44:30.554522 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.554524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.555496 (+ 972us) server_negotiation.cc:407] Connection header received
0806 22:44:30.555604 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.555605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.555615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.555645 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.556102 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55990: BlockingRecv error: recv got EOF from 127.0.0.1:55990 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 22:44:30.557000 3438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.554864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.0.186.254:46763)
0806 22:44:30.554902 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.554904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.556421 (+ 1517us) server_negotiation.cc:407] Connection header received
0806 22:44:30.556528 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.556529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.556540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.556583 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.556938 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50822: BlockingRecv error: recv got EOF from 127.0.0.1:50822 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:44:30.557950 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.555203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36750 (local address 127.0.186.253:42237)
0806 22:44:30.555240 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.555241 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.557262 (+ 2021us) server_negotiation.cc:407] Connection header received
0806 22:44:30.557428 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.557430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.557439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.557465 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.557892 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36750: BlockingRecv error: recv got EOF from 127.0.0.1:36750 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:44:30.560340 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.559598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.0.186.253:42237)
0806 22:44:30.559632 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.559634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.559850 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:44:30.559950 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.559951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.559968 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.559994 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.560294 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36766: BlockingRecv error: recv got EOF from 127.0.0.1:36766 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:44:30.563301 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.562564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.0.186.253:42237)
0806 22:44:30.562596 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.562598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.562817 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:44:30.562906 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.562907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.562917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.562942 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.563231 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36772: BlockingRecv error: recv got EOF from 127.0.0.1:36772 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:44:30.567093 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.566376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.0.186.253:42237)
0806 22:44:30.566407 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.566409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.566634 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:44:30.566725 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.566725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.566736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.566762 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.567052 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36782: BlockingRecv error: recv got EOF from 127.0.0.1:36782 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 22:44:30.572925 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.572188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.0.186.253:42237)
0806 22:44:30.572222 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.572224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.572435 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:44:30.572540 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.572541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.572550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.572577 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.572883 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36788: BlockingRecv error: recv got EOF from 127.0.0.1:36788 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:44:30.582912 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.582052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36792 (local address 127.0.186.253:42237)
0806 22:44:30.582086 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.582087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.582310 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:44:30.582402 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.582403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.582412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.582439 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.582866 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36792: BlockingRecv error: recv got EOF from 127.0.0.1:36792 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:44:30.601217 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.600269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.0.186.253:42237)
0806 22:44:30.600313 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.600315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.600614 (+ 299us) server_negotiation.cc:407] Connection header received
0806 22:44:30.600725 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.600726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.600738 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.600769 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.601161 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36804: BlockingRecv error: recv got EOF from 127.0.0.1:36804 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:44:30.635608 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.634615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.0.186.253:42237)
0806 22:44:30.634659 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.634661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.634956 (+ 295us) server_negotiation.cc:407] Connection header received
0806 22:44:30.635064 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.635065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.635076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.635105 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.635548 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36810: BlockingRecv error: recv got EOF from 127.0.0.1:36810 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:44:30.701560 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.700945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.0.186.253:42237)
0806 22:44:30.701004 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.701005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.701094 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:44:30.701144 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.701144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.701154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.701216 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.701512 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36814: BlockingRecv error: recv got EOF from 127.0.0.1:36814 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 22:44:30.831383 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:30.830720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.0.186.253:42237)
0806 22:44:30.830781 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:30.830783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:30.830874 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:44:30.830927 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:30.830928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:30.830938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:30.830993 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:30.831329 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36822: BlockingRecv error: recv got EOF from 127.0.0.1:36822 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:44:31.088871 3901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5344751f9ea64a2daff53956e26f0682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5344751f9ea64a2daff53956e26f0682" member_type: VOTER last_known_addr { host: "127.0.186.254" port: 46763 } } peers { permanent_uuid: "ffc3e4c18a6d423abdeb073706353749" member_type: VOTER last_known_addr { host: "127.0.186.253" port: 42237 } } peers { permanent_uuid: "2b7d90abc7bf4b90bf4ed6b9ab386b6f" member_type: VOTER last_known_addr { host: "127.0.186.252" port: 38351 } } }
I20260806 22:44:31.089020 3901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5344751f9ea64a2daff53956e26f0682 [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:31.089263 3564 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:31.088525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.0.186.253:42237)
0806 22:44:31.088605 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.088607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.088626 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:44:31.088716 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.088717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.088728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.088787 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:31.089215 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36830: BlockingRecv error: recv got EOF from 127.0.0.1:36830 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:44:31.459654 3718 catalog_manager.cc:1259] Loaded cluster ID: bc1ba5fae59d4aa6be0bbafa2ba39cd9
I20260806 22:44:31.459729 3718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: loading cluster ID for follower catalog manager: success
I20260806 22:44:31.460639 3718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: acquiring CA information for follower catalog manager: success
I20260806 22:44:31.460932 3718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b7d90abc7bf4b90bf4ed6b9ab386b6f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:31.530544 3785 catalog_manager.cc:1259] Loaded cluster ID: bc1ba5fae59d4aa6be0bbafa2ba39cd9
I20260806 22:44:31.530622 3785 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: loading cluster ID for follower catalog manager: success
I20260806 22:44:31.531456 3785 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: acquiring CA information for follower catalog manager: success
I20260806 22:44:31.531761 3785 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ffc3e4c18a6d423abdeb073706353749: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:44:31.605072 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:31.602568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.0.186.253:42237)
0806 22:44:31.602697 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.602699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.602714 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:44:31.602780 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.602780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.602831 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.602933 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:31.603051 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.603577 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.604365 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.604575 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.604634 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:31.604749 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:31.604860 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:31.604943 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:31.604955 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 22:44:31.604975 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260806 22:44:31.608835 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:31.606761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.0.186.252:38351)
0806 22:44:31.606866 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.606867 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.606874 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:31.606911 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.606912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.606946 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.607018 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:31.607101 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.607616 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.608266 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.608453 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.608499 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:31.608599 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:31.608694 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:31.608742 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:31.608759 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 22:44:31.608775 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260806 22:44:31.612893 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:31.610374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.0.186.252:38351)
0806 22:44:31.610411 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.610413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.610527 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:44:31.610614 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.610614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.610623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.610665 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:31.610742 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.611296 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612198 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.612474 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612535 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:31.612668 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:31.612697 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:31.612721 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:31.612807 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:44:31.612826 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:44:31.613157 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:31.610300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.0.186.253:42237)
0806 22:44:31.610335 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.610337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.610514 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:44:31.610601 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.610602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.610612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.610656 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:31.610833 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.611596 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612464 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.612734 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612787 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:31.612911 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:31.612942 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:31.612981 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:31.613063 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:44:31.613089 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:44:31.613790 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:31.610324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.0.186.254:46763)
0806 22:44:31.610641 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:31.610643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:31.610650 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:44:31.610701 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:31.610702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:31.610752 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:31.610839 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:31.610959 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.611722 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612629 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:31.612910 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:31.612970 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:31.613460 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:31.613548 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:31.613605 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:31.613691 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:44:31.613712 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":54,"threads_started":1}
I20260806 22:44:31.615217 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3330
I20260806 22:44:31.627012 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3448
I20260806 22:44:31.635746 747 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwWuo9B/build/release/bin/kudu with pid 3569
2026-08-06T22:44:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2880 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5808 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274404 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST